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THE STANDING COMMITTEE ON 
INDUSTRY, SCIENCE AND TECHNOLOGY 


has the honour to present its 
TENTH REPORT 


Pursuant to Standing Order 108(2), the Standing Committee on Industry, Science 
and Technology proceeded to a study on the three federal granting agencies, peer review 
funding, and the Canada Research Chairs Program. After hearing evidence, the 
Committee agreed to report to the House as follows: 


CHAIR’S FOREWORD 


Prosperity in the new knowledge-based economy depends, in large part, on 
innovation. One of the major priorities of the federal government is to improve Canada’s 
Innovation performance since Canada lags behind other advanced economies in this 
regard. The federal government’s Innovation Strategy, released in February 2002, is 
intended to serve as a blueprint for increasing innovation and productivity in Canada. The 
Innovation Strategy outlines broad goals, targets and federal government priorities to help 


meet the objective of improving Canada’s innovation performance and standard of living 
over the next decade. 


Two of the areas listed in the Strategy that influence innovation capacity are “skills” 
and “knowledge performance.” The Strategy indicates that the federal government's role 
in improving capacity in these areas will be achieved, in part, via an increase in the 
federal government's current investments in research and development (R&D), and in the 
training of highly qualified personnel. Part of those investments are in the three federal 
granting agencies, the Natural Sciences and Engineering Research Council of Canada, 
the Social Sciences and Humanities Research Council of Canada and the Canadian 
Institutes of Health Research. 


The Committee believes that these three agencies all have vital roles to play in 
improving Canada’s innovation capacity through their support of research, training, and 
knowledge and technology transfer. However, earlier work conducted by the Committee 
suggested that improvements in how the agencies allocate federal research funds may 
be necessary. In the Committee’s June 2001 report A Canadian Innovation Agenda for 
the Twenty-first Century, the Committee committed to examining the mandates, 
processes and decision-making criteria of the agencies to ensure that funds are being 
managed in the best possible way before additional investments are made. Part of the 
Committee’s concerns about the allocation of federal research funds centred on the low 
research capacity of certain small and regional universities, and the relatively low success 
rates of these universities’ applicants in the granting agencies’ competitions. The 
Committee believes that small research institutions have an important role to play in 
ensuring that all regions of Canada participate in and benefit from the knowledge-based 
society that we are building; access to federal granting agency funds for researchers at 


these institutions is critical in this regard. 


The Committee undertook an examination of these issues by organizing a series 
of roundtables in which representatives of the granting agencies, advisory bodies, policy 
groups, university and college associations, and the research community expressed their 
opinions on peer review and other issues related to the allocation of federal research 
funds. A wide range of views on this topic was heard; the Committee believes that this 
report captures the essence of those views and that its recommendations will help the 


granting agencies in their work of supporting R&D and innovation. 


vii 


| would like to thank the individuals and organizations that took part in this series of 
hearings, and express my appreciation to them for helping the Committee in its ongoing 
work to improve Canada’s innovation performance. The Committee, in the context of its 
planned future work on innovation and productivity, will continue to study the allocation 
and management of federal R&D funds. With the help of Canada’s research community, 
the Committee will continue to promote the importance of science, technology and 
innovation for improving Canada’s economic performance and the quality of life of its 


citizens. 


viii 


TABLE OF CONTENTS 


GAIRS FOREWOR Dr srs. 4 ot OS IPS ae vii 
RECOMMENDATIONS:.:..:.:444.00 ue eee ete ee career ae, ER AN Ip xi 
INTRODUCTION: RE Re RE ne even 1 
CHAPTER ONE: AN OVERVIEW OF THE FEDERAL GRANTING AGENCIES 
AND THE CANADA RESEARCH CHAIRS RROGRAMASS semer 3 
the, Federal, Granting Agencies.…... Zi. 4 
Budgets and Agency Objectives i... ee ee 4 
Peer Review ane EEE ee ee en eee ee ee 6 
The Canada Research Chairs Programs PRE ee Es (i 
Buaget and"Program Goals LL CR oe se fe 1 
AHOCALION OF CAAGITS res rnreases eme eee ee tee: Ete ee eae 8 
GCNAIES OIG CUON IP TOCOSS: sis teats idee M D eT 9 
Evaluation Of the ProGrail, c..icscenaecesasseh eee cles Re ee 10 
CHAPTER TWO: SMALL AND REGIONAL UNIVERSITIES AND 
THE ALLOCATION OF FEDERAL RESEARCH FUNDS .….................................. 11 
Funding of Researchers at Small Universities…................................................. 11 
Roots of the Funding Problem for Researchers at Small Institutions ........0000 14 
Increasing Research Capacity at Small Institutions... 15 
Payment ofilndiréctCosisiss. 2 Re RE PR RE D Os oo 15 
Programs Directed to Small and Regional Institutions ..........c:cccscccssceseesneseesseeees 17 
CHR gk he stv a tomaehirg date vive'v bern is ob rade Secs Spe URE ae oe ee PE 18 
SCAR raced ae cee es CCEA as ge re ag ee 19 
NSERC cei NSE RE OR I PRET 20 
Focusing on Research Strengths...…......................................….....#vt 20 
Increased Funding for Research in the Social Sciences... 21 
Increased Funding to All Three Federal Granting Agencies ..............:cceeeeeesee 23 
Potential Bias Against Small Institutions in Decision-making Processes..................... 24 
The Canada Research Chairs Program and Small Universities …............................. 25 


CHAPTER THREE: EXCELLENCE, RELEVANCE AND STRATEGIC 


RE SE ARGH sivvssctvccis reenter ccenertdracwenes code nee eens tal AT 0 OI A RS a OR RE a 
Relevance and “Strategic” Granting Programs... 27 
Research in Interdisciplinary and Emerging Areas .…................................................ 30 
High-risk Research sense 32 
Increased Collaboration Among the Granting Agencies 33 
Basic and “Curiosity-driven” Research ss 34 
Federal'Researcn PAOTIUIES Mec denee cos sans tons cnesenane eee ten en 35 
CHAPTER FOUR: OTHER PLAYERS IN CANADA’S INNOVATION 

FRAME WORK esses. eves tenn dannstovedecdi niountertoobedsnnodaamandsdt ates tele OR TS 39 
Highly Qualified Personnels ns RE Mra eee 39 
Collèges: anses nantais EE Me 43 
CHAPTER FIVE: IMPROVEMENTS TO THE SYSTEM FOR 

ALLOCATING FEDERAL RESEARCH FUNDS..................................................,. 47 
Perceived Weaknesses in the System... CR 47 

Inadequate or Inconsistent Feedback to Applicants..…......................…................... 47 

Peer Review Is Untested rs rates es enter conh oer ania ee ene eee 50 

Internal Evaluation of Agency Programs and Practices May Be Insufficient.......... où 

Peer Reviewer Overload int:s. fines. LE de en rs PP OR CEE D 52 

Outcomes and Impact of Research Are Not Adequately Measured and 

FI@DOMOG A sweat 202 acuay cas Le SALE; case uN D UD dune ane es RE TE 54 
Altematives.to Peer REVIEW. ee RS ee of 

BIDIOMONICS EE TR PR race ates atttortonetaaaehineriet cic cigs eee cd Re 58 

Bicameral FReVIeW..x.. 5,45 Re RER Re ee 59 

FTOQUCHVIy-DaSech FOUR EUNOINO tee cae et se ee ee eee 59 
CONCLUSIONS. a shtee. nent RON a EN ee 61 
APPREND RS Se nu SRI oe 63 
APPENDIX 252. ee RS NON TOR RO 65 
APPENDIRS Sr rtsieur naar ra sists cot pet tn de NE a ee 69 
APPENDIX4 — WITNESSES ee NT RE 73 
REQUEST FOR GOVERNMENT RESPONSE eee. 75 
DISSENTING OPINION — Canadian Alliance .......0......ccccccccccccccsccsccsccecccsececeeseeseeees Lf 
MINUTES OF PROCEEDINGS 7... en ee eee 79 


RECOMMENDATIONS 


RECOMMENDATION 1 


That the Government of Canada, in consultation with the provinces 
and territories, instate a permanent program for the support of the 
indirect costs of federally funded research in its next budget, and that 
the formula used to calculate the grant to each institution take into 
account the differential costs between large and small institutions. 


RECOMMENDATION 2 


That the Government of Canada, in consultation with the provinces 
and territories, set up targeted programs through the granting 
agencies to assist “underdeveloped” small and regional institutions 
strengthen their research capacity in key areas, and that 
appropriations be made to the agencies to fund these programs. 


RECOMMENDATION 3 


That the Government of Canada increase the level of funding to 
SSHRC, especially given the impact of this funding on researchers at 
small institutions. 


RECOMMENDATION 4 


That the Government of Canada immediately increase funding to all 
three granting agencies so that success rates and/or grant levels for 
excellent research can be increased. 


RECOMMENDATION 5 


That the Government of Canada encourage the granting agencies to 
collaborate further with each other and with non-academic 
stakeholders, including the provinces and territories, when deciding 
on target areas within strategic programs. The agencies should also 
ensure that formalized review mechanisms and adequate levels of 
funding are available to support high-calibre interdisciplinary research 
or research in emerging areas, especially that which crosses the 
boundaries of the agencies’ mandates. The agencies should consider 
establishing a formal mechanism to facilitate collaboration on all 


XI 


levels so that their operations are more uniform, allowing for 
“seamless” access by the research community to federal research 


funds. 


RECOMMENDATION 6 


That the Government of Canada establish a more formalized 
mechanism, in consultation with the provinces and territories, for 
setting or modifying S&T policy, deciding on funding priorities, and 
ensuring that they are implemented. Such a framework could include 
a science advisory body and/or Chief Scientific Adviser that would 
report directly to Parliament. 


RECOMMENDATION 7 


That the Government of Canada, given the increases in overall funding 
levels that should be made to the agencies, encourage the three 
granting agencies to increase success rates and stipend levels for 
existing graduate scholarship and postdoctoral fellowship programs. 
The granting agencies should also create or enhance scholarship 
programs that introduce undergraduate students to research. 


RECOMMENDATION 8 


That the Government of Canada create separate research funding 
programs administered either by the granting agencies and/or other 
government bodies to support college researchers and students. The 
programs’ selection criteria should reflect the situation at colleges 
and be based on excellence. Appropriations should be made to the 
relevant agencies to deliver these programs. 


RECOMMENDATION 9 


That the Government of Canada ensure that the granting agencies 
release all information on file relevant to a funding recommendation to 
applicants in addition to the notification of decision. Additionally, a 
formal appeal process, limited to perceived errors in procedure or 
fact, should be in place for applicants to all peer-reviewed programs, 
and a third-party, not the original selection committee, should review 
appeals of decision. 


Xii 


RECOMMENDATION 10 


That the Government of Canada require the granting agencies to 
engage in more regular internal reviews of their own programs and 
practices (including peer review), and to periodically examine 
decision-making processes at other Canadian and foreign agencies to 
ensure that best practices for the allocation of research funds are in 
place. The results of these internal evaluations should be easily 
accessible to the research community and general public. 


RECOMMENDATION 11 


That the Government of Canada ensure that the federal granting 
agencies take steps to better measure and report on the outcomes 
and, where possible, impacts of their research programs for the 
benefit of the general public. 


xiii 


ea. de Oe 


6; 
1:70 A RL Are sot 
Pate” aii à 

shy in a 


| te is een y" " 
ue pe i TEA AR 4 
+ en FANS 
* CEs ‘a tp es 


Mis DUT is 
' se at rs de 

; | ie ital tuée | PTT. 
CR ee ee sat | 4 


Peut Boer eae Pe TRS PR re, er TA i 
an a ve - Lt 
re PARUS: CR x ENS NE x im, LA 


= ae | pany oa À } ay … Ere t Hy je À ve f 
9 Vv à y ih ALP yp : & ‘fi 


DS dit 


tp] 


AT othe ao ai Qe ee 


LU ae à ti 
‘ awa | À 
ae . t or vy 
i] i s i 
’ SALE aye + 
; i 3 ° ns { 
Far > : Le 


INTRODUCTION 


In Its report, A Canadian Innovation Agenda for the Twenty-first Century’ 
released in June 2001, the Committee expressed some broad concerns about how 
research funds are allocated by the three federal research granting agencies (the 
Natural Sciences and Engineering Research Council of Canada or NSERC, the Social 
Sciences and Humanities Research Council of Canada or SSHRC, and the Canadian 
Institutes of Health Research or CIHR). Although generally supportive of the agencies’ 
work and accomplishments, it was concerned about how the agencies carry out their 
mandates and about the decision-making processes employed for funding research 
projects and programs. The Committee indicated that given the government’s 
commitment to double Canada’s annual investment in research and development 
(R&D) by 2010, a review of decision-making processes and selection criteria used by 
the granting agencies is important, before additional investments are made, to ensure 
that funds are being managed in the best possible way. 


The Committee detailed some specific concerns in its report about the allocation of 
federal research funds by the granting agencies. In particular, the Committee was 
troubled to learn of the relatively weak research capacity of small universities and the 
corresponding low success rates of researchers from small universities in obtaining 
research grants from the federal granting agencies. The Committee was concerned about 
the concentration of federal research funds in a few large institutions. The Committee 
also voiced its concerns about the method used to calculate the allocation of Research 
Chairs to universities under the Canada Research Chairs Program announced in the 
federal government’s 2000 budget; specifically, it was worried that the allocation of the 
Chairs based on the past performance of universities in obtaining federal research grant 
funds would reinforce the existing disparity in research capacity across the country. The 
Committee was also concerned that the Program would lead to the “poaching” by large 
universities of the best faculty from smaller universities thus augmenting even further the 
disparity in research capacity. On the subject of research capacity, the Committee 
believes that the government is not doing enough to develop this capacity in certain 
regions of the country and in smaller institutions. 


Chapter One provides an overview of the budgets, missions and grant selection 
processes at the three granting agencies and in the Canada Research Chairs Program. 
All three of the granting agencies use a similar peer-review process in which experts 
assess the quality of research proposals and make funding recommendations to the 


agencies. 


In Chapter Two, the Committee addresses the concerns of small and regional 


institutions with respect to the allocation of federal research funds. The Committee details 
ee LO Se 
: The report can be accessed electronically at: 


http://www. parl.gc.ca/InfoComDoc/37/1 /INST/Studies/Reports/indu04-e.htm 


concerns about the weak research capacity of small universities, and discusses 
suggestions made by witnesses as to how research capacity can be improved at these 
institutions. Potential mechanisms to improve research capacity include establishing a 
permanent program for the payment of the indirect costs of research by the federal 
government, providing programs through the granting agencies that are targeted to 
building research capacity in small or regional universities, ensuring that small and 
regional universities find their research “niches,” increasing funding levels for research in 
the social sciences and humanities, and increasing funding levels to all three granting 
agencies. Each of these issues is examined in this chapter, and the Committee also 
addresses the matter of whether there are any inherent biases against researchers from 
small institutions in the decision-making processes of the federal granting agencies. 


The issue of funding priorities for Canada and their relationship to agency 
programs and selection criteria is discussed in Chapter Three. For the large majority of 
granting agency programs, the excellence of the proposed research and the researchers 
is the most important criterion for selecting which research proposals are to be funded. 
For other programs, the socio-economic relevance of the proposed research in target 
areas of national importance is also an important selection criterion; in a few cases, it is 
as important as the excellence of the proposed research. The issue of granting agency 
selection criteria and what proportion of federal government research funds should be 
directed towards supporting research in target areas is addressed in this chapter. 
Additionally, the Committee addresses the matter of support for interdisciplinary research 
and research in emerging areas, and suggests that the agencies can make improvements 
in these areas. The chapter also discusses federal policy in the context of funding 
research in strategically important areas and indicates that stronger direction in this area 
is required. 


Chapter Four addresses the important contributions that highly qualified personnel 
and college researchers make to R&D in Canada’s knowledge-based economy. The 
Committee believes that, for different reasons, both groups are undervalued in the 
present system for allocating federal research funds. Mechanisms to improve the 
situation for both groups are presented in this chapter. 


The Committee believes that although the present system for allocating limited 
federal research funds is the most appropriate and effective mechanism available, there 
is certainly room for improvement on several levels. In Chapter Five, the Committee 
recommends that improvements can be made to: the monitoring and evaluation of peer 
review practices by the agencies; the types of feedback and appeal mechanisms 
provided to applicants; and the measurement and communication of the outputs, 
outcomes and impacts of federally funded research. In addition, the Committee discusses 
alternatives to peer review, and the feasibility of such proposals. 


CHAPTER ONE: AN OVERVIEW OF THE FEDERAL 
GRANTING AGENCIES AND THE CANADA RESEARCH 
CHAIRS PROGRAM 


Research and development (R&D) may lead to innovation.* Even if R&D fails to 
deliver innovation in the short term, it adds to the general body of knowledge and can 
point to other, often more promising, lines of enquiry that may lead to innovation in the 
longer term. Innovation is acknowledged to be a basic building block of economic growth. 
In fact, according to the Organisation for Economic Co-operation and Development, 
innovation and technological change have become the principal drivers of growth in 
advanced economies.® The results of R&D may also lead to other (e.g., social, 
environmental or medical) advances that may not lead directly to wealth creation but 
improve the quality of life of the world’s citizens. 


Governments around the world provide support for R&D. A large body of economic 
literature” exists to explain the rationale behind the investment by government in R&D. 
According to contemporary economic theory, the benefits of R&D extend beyond the 
performers of the R&D themselves (or “spill over’) to other sectors of the economy. 
Econometric analyses suggest that the social rates of return to R&D investments can be 
up to five times higher than private rates of return, and that social rates of return on basic 
R&D are higher than those on applied R&D. The value of the benefits from investments in 
R&D is not completely appropriable by the R&D performers, and in a market economy, 
“inappropriability” of value leads to underproduction (i.e., the market fails to allocate an 
efficient quantity of resources to R&D). Governments support R&D to compensate for the 
market’s socially sub-optimal investment in R&D, and to rectify shortcomings in the 
government's ability to provide rights to the R&D performer for appropriations resulting 
from those investments. Government support for R&D can take several forms including 
the provision of grants, prizes, tax incentives and an intellectual property rights regime. 


The Government of Canada spent $6.85 billion (or 4.4% of the federal 
government’s total budget) on science’ and technology (S&T) activities for fiscal year 
2000-2001. Of this amount, $4.21 billion was spent on the intramural performance and 


2 Innovation is defined as “a process through which economic value is extracted from knowledge through the 
generation, development and implementation of ideas to produce new or improved products, processes and 
services.” Conference Board of Canada, /nvesting in Innovation: Third Annual Innovation Report (2001). 

3 OECD, OECD Science, Technology and Industry Outlook 2000. 

4 See references in G. Lenjosek and M. Mansour, “Why and How Governments Support R&D,” Canadian Tax 
Journal, Vol. 47, 1999, p. 242-272. 

5 The Committee uses the term “science” to represent the whole ranges of sciences, including social sciences, 
natural sciences, engineering and health sciences. 


extramural funding of R&D.° For fiscal year 2001-2002, estimates indicate that the 
government spent $7.39 billion on S&T activities (4.5% of its total budget), of which $4.65 
billion was on R&D funding. Canada’s Gross Domestic Expenditures on R&D (GERD)’ in 
2001 ($20.871 billion’) were estimated to be about 1.9% of its GDP. In 2001, the 
percentage of the GERD financed by the federal government was estimated to be 
approximately 18%° or about 0.35% of the GDP. In 1999, the last year for which 
comparative international figures are available, Canada’s GERD as a percentage of its 
GDP (1.77%) lagged behind that of all other G-7 countries with the exception of Italy. 

The three granting agencies, the Natural Sciences and Engineering Research Council 
(NSERC), the Social Sciences and Humanities Research Council (SSHRC), and the 
Canadian Institutes of Health Research (CIHR), "' as well as the Canada Foundation for 
Innovation, are the major federal government funders of R&D performed in Canadian 


universities. 7 


The Federal Granting Agencies 
Budgets and Agency Objectives 


In fiscal year 2000-01, slightly more than $1 billion flowed through the three federal 
granting agencies — NSERC, SSHRC and CIHR — to fund university-based research, to 
train highly qualified personnel and to foster research partnerships among academia, 
government institutions and the private sector. By 2002-03, projected base budgets for 
the three agencies are expected to total $1.28 billion. Over the last 10 years, funding to 
the three agencies has increased substantially, with the largest percentage increase 
going to CIHR, which has seen its budget more than double over that period 
(see Table 1). 


According to international convention, Statistics Canada splits S&T activities into R&D and related scientific 
activities. The latter includes such activities as data collection, information services (e.g., libraries and 
museums), and special services and studies (e.g., testing and standardization, and feasibility studies). 


GERD = total intramural expenditures on R&D performed on the national territory during a given period. It 
includes R&D performed within a country and funded from abroad but excludes payments made abroad for R&D 


(OECD definition). The Federal Government values that are part of GERD are its R&D activities performed 
intramurally. 


Statistics Canada, Federal Scientific Activities 2001-2002e, April 2002. 
° Ibid. 
Statistics Canada, Service Bulletin Science Statistics, Vol. 25, No. 8, November 2001 (see figure in Appendix 1). 


The two granting councils, NSERC and SSHRC, report to the Minister of Industry, and CIHR reports to the 
Minister of Health. 


Statistics Canada, Federal Scientific Activities 2001-2002e, April 2002. 


Table 1. Base Budgets for the Three Federal Granting Agencies 


(NSERC, SSHRC and CIHR) '° 
RU er ee ee 


Fiscal Year 
ae SE OS eS er "0 ee ee, De 
Agency 1990-91 1995-96 2000-01 2001-02 2002-03 


NSERC $424 million $443 million $521 million $528 million $559 million 
SSHRC $90 million $100 million $128 million $134 million $157 million 


CIHR $243 million $252 million $360 million $487 million $562 million 
EE RE OE DE ene Venn Lola: hr Toca 


The Natural Sciences and Engineering Research Council of Canada is the national 
instrument for making strategic investments in Canada’s capabilities in S&T. Its mission is 
to invest in people, discovery and innovation to build a strong Canadian economy and to 
improve the quality of life for all Canadians. The agency achieves its mission by awarding 
scholarships and research grants through peer-reviewed competition and by building 
partnerships among universities, colleges, governments and the private sector. NSERC 
supports nearly 16,000 students and fellows in their advanced studies and funds more 
than 9,700 researchers. In addition, over 500 companies participate in some of NSERC’s 
programs. 


The Social Sciences and Humanities Research Council of Canada is Canada’s 
federal funding agency for university-based research and graduate training in the social 
sciences and humanities. SSHRC’s goal is to help build the human knowledge and skills 
Canada needs to improve the quality of its social, economic and cultural life. The agency 
offers grants for basic and applied research by Canadian university-based researchers 
and scholars. It also offers fellowships for research training at the doctoral and 
postdoctoral levels, and grants to support the publication and dissemination of research 
findings. Additionally, it is involved in a range of efforts to help integrate social sciences 
and humanities research expertise into the process of social and economic 


policy-making. 


The Canadian Institutes of Health Research is Canada’s federal agency for health 
research. Its objective is to excel, according to internationally accepted standards of 
scientific excellence, in the creation of new knowledge and its translation into improved 
health for Canadians, more effective health services and products and a strengthened 
health care system. The CIHR concept involves a multi-disciplinary approach, organized 
through a framework of 13 “virtual” institutes, each one dedicated to a specific area of 


18 Base budget (to the nearest $1 million) = grants and scholarships (or awards), and operating expenses. Figures 
reported here (and elsewhere in the report, unless stated otherwise) do not include so-called “flow through 
funds” allocated to each agency for the Canada Research Chairs Program and the Networks of Centres of 
Excellence Program. Operating expenses average 6.3%, 9.2%, and 6.4% of base budgets for NSERC, SSHRC 
and CIHR, respectively for fiscal years 2000-01 through 2002-03 (includes NCE operating expenses). Figures 


supplied by the agencies. 
14 Data from 1990-91 and 1995-96 are for the Medical Research Council. 


focus, linking and supporting researchers pursuing common goals. An Act of Parliament 
created CIHR in 2090, replacing the former Medical Research Council. 


Peer Review 


With some modifications, all three granting agencies use a similar peer-review 
process for making funding recommendations and allocating limited research funds. In 
peer review, experts from a particular field of study assess the quality of (depending on 
the context) research proposals, scholarly manuscripts, or other bodies of work in that 
field or in related fields. “Non-peers” (e.g., administrative staff or potential end-users of 
the research results) may also be members of peer review committees. The results of the 
evaluation may be used to determine, among other things, whether a researcher is 
granted funding, a manuscript is published in a scholarly journal, or a professor receives 
tenure. The first recorded use of peer review as a mechanism to assess the quality of 
scientific contributions is associated with the founding of the Philosophical Transactions of 
the Royal Society in 1665."° The formal use of peer review as a mechanism for the 
allocation of central government research funds has its origins in the middle of the 
20th century when structured government programs for the support of R&D were 
established in many countries. '° 


Peer review has been used by the Canadian federal granting agencies to 
determine the allocation of research funds since the agencies’ creation (NSERC in 1978, 
CIHR in 2000 (replacing the Medical Research Council, created in 1965), and SSHRC in 
1977). The quality of each proposal is judged according to selection criteria that are 
established by the agencies (usually after consultation with members of the community). 
The selection criteria generally include some combination of the quality of the proposal 
and the investigator(s), and the past achievements of the applicant(s). In some cases, the 
short-term socio-economic relevance of the proposed work, often in particular “target” or 
“strategic” areas, is also evaluated. Depending on the program in question, peer review 
may be conducted by a panel of experts set up by the agency, by external reviewers, or 
by some combination of both methods. 


The experts who sit on the selection panels or committees are chosen according 
to guidelines set up by the granting agencies. These guidelines are intended to ensure 
that the committees are proportionally balanced in terms of gender, regional, institutional 
and “sectoral” (university, government or industry) representation, and in language 
abilities. At CIHR, for its strategic or “thematic” funding areas, members who are not 
researchers are also included on committees: 


Fe [Ojther experience and expertise can add important additional dimensions and 
perspectives to the review of applications. For example, we include community 


'° D.E. Chubin and E.J. Hackett, Peerless Science: Peer Review and U.S. Science Policy, State University of New 


York Press, Albany, New York, 1990. p. 19. 


F.Q. Wood, The Peer Review Process. Report commissioned for the National B i 
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members on committees that review proposals in the area of Aboriginal people’s 
health. [Mark Bisby, CIHR, 39:15:35] 


Success rates vary among agencies and among programs at each agency, but the 
demand for funding outstrips the amount of funds available at each agency: 


While about 70% of the 5,000 proposals that are reviewed each year are considered 
worthy of funding, the budget that is available allows only about 30% to be funded. 
The CIHR’s rating scale, which goes from 0 to 5, currently requires about a 4, or an 
excellent rating, on the rating scale for a proposal to be sure of being funded. 
[Mark Bisby, CIHR, 39:15:30] 


For their major funding programs, each agency has a different philosophy in how limited 
funds are allocated. At NSERC, a large proportion (about 80%) of applicants to its 
Discovery Grants Program (approximately $255 million, or 49% of NSERC’s grants and 
scholarships budget in fiscal year 2000-01) is funded, but the size of the grant is 
relatively small, averaging about $37,000 per year over a four-year period for the 
2001 competition. At SSHRC a relatively small proportion, approximately 40%, of 
applicants to the Standard Research Grants Program (approximately $39.3 million or 
31% of SSHRC’s grants and scholarships budget in 2000-01) is funded and the 
average research grant is just under $25,000 per year over three years. At CIHR, the 
success rate is relatively small, about 30%, in its Operating Grants Program 
(approximately $207.5 million or 58% of CIHR’s grants and awards budget in 2000-01), 
but the grant size is relatively large, averaging $100,000 per year for a three- to five- 
year period. 


The agencies argue that peer review is the most efficient way of allocating 
research funds in light of the limited monies available for research funding: 


Like many agencies around the world, we believe peer review is the best way of 
selecting the highest quality applications from thousands of competing proposals. 
[Elizabeth Boston, NSERC, 39:15:40] 


First of all, in Canada, peer review is already more than 40 years old and it is a 
process that is recognized all over the world. Given meagre resources, it remains 
the best system to distribute public funds. However, it is a system that first and 
foremost ensures a high degree of excellence in subsidized research. It is the most 
independent, the most transparent and the most objective granting process. 


[Ned Ellis, SSHRC, 39:15:40] 


The Canada Research Chairs Program 


Budget and Program Goals 


In its 2000 budget, the Government of Canada provided $900 million to support 
the establishment of 2000 Canada Research Chairs in universities across the country by 


2005. About 400 new Chairs, recruited from both inside and outside Canada, will be 
named in each year. 


The main goals of the Canada Research Chairs Program (CRCP) are to enable 
Canadian universities, together with their affiliated research institutes and hospitals, to 
achieve the highest levels of research excellence and to become world-class research 
centres in the global, knowledge-based economy. The Program’s goals are to be 
achieved through the creation of Chairs in the natural sciences, engineering, health 
sciences, social sciences and humanities. The Program’s emphasis is on investment in 
basic and applied research at Canada’s universities. 


Under the Program, there are two types of Chair: (1) seven-year renewable “Tier I” 
Chairs targeted at experienced researchers who are acknowledged by their peers as 
world leaders in their own fields (each Chair is worth $200,000 annually); and (2) five-year 
“Tier ||” Chairs, renewable once, targeted at researchers who are acknowledged by their 
peers as having the potential to lead in their fields (each worth $100,000 annually). 


The CRCP is governed by a Steering Committee comprising the presidents of 
NSERC, CIHR, SSHRC and the Canada Foundation for Innovation, as well as the Deputy 
Minister of Industry Canada. 


Allocation of Chairs 


Each eligible!” university receives a predetermined allocation of Chairs per year. 
The allocation is proportional to the amount of eligible grant funding a university has 
received from the three federal granting agencies, including funds received by any 
affiliated research institutes and hospitals, over a three-year period. A three-year “moving 
average” is used to calculate the allocation, so that the number of Chairs allocated is 
adjusted annually to reflect changes in granting agency funding received by each 
university. 


A “special allocation” (6% of the 2000 Chairs) has been reserved for smaller 
universities, i.e., those universities that have received one per cent or less of the total of 
federal research granting agency funds over the three-year period. Depending on the 
amount of research funding they have received from the granting agencies, these smaller 
institutions may: 1) qualify for both a regular allocation (as above) and a special 
allocation; 2) qualify only for a special allocation; or 3) not be eligible in the first three 
years of the Program (if they received less than an average of $100,000, in total, per year 
from the federal granting agencies). Over the first three years of the Program, the special 
allocation will be distributed as follows: a) universities that have received, on average, 
between $100,000 and $200,000 per year from the three federal granting agencies 


7 Only Canadian degree-granting universities are eligible to participate in the Program. During the first three years 


of the Program, these institutions are eligible only if they have received, annually, an average of $100,000 or 
more from the three federal granting agencies. | 


combined will receive a special allocation of one Tier | Chair or equivalent ($200,000). 
This allocation was made available in the first year; or b) universities that have received, 
on average, at least $200,000 per year, but less than one per cent of the funding from the 
three granting agencies combined, will receive a special allocation of $400,000 for Tier | 
or Tier Il Chairs. In the fourth and fifth years of the Program, there are plans to hold a 
competition for the Chairs remaining in the reserve. Universities eligible to compete will be 
those that have been awarded one percent or less of the total funding from the three 
research granting agencies combined. 


The 1880 Chairs allocated to the larger universities are divided among discipline 
groups as follows: 1) for the natural sciences and engineering, 45 percent or 846 Chairs, 
over five years; 2) for health, 35 percent or 658 Chairs, over five years; and 3) for the 
social sciences and humanities, 20 percent or 376 Chairs, over five years. The 
percentage of Chairs allocated to a university in each of these three discipline groups is 
the same as the percentage of relevant research agency eligible funds that the university 
received over the three-year period. In terms of the distribution of Chairs by tier, for 
universities receiving a total allocation of one Chair, an amount of $200,000 will be set 
aside. For universities receiving an allocation of more than one Chair, half of the 
allocation will be based on Tier 1 ($200,000) and half on Tier 2 ($100,000). Small 
institutions that have received Chairs through a “special allocation” have some flexibility in 
how they allocate their Chairs (by tier and by granting agency). 


Chair Selection Process 


The Program requires that each participating institution submit a “Strategic 
Research Plan” describing how the Chairs will be deployed. Chairs are assigned to 
priority areas identified by universities in their plans, and are filled by individuals who meet 
the Program’s criteria of excellence. Appointment of the Chairs is based on nominations 
from Canadian universities. Appointments follow a peer-review process conducted by 
members of the Program’s College of Reviewers and, where necessary, the 
Interdisciplinary Adjudication Committee. Both the College and the Adjudication 
Committee are composed of some of the world’s leading experts in disciplines being 
funded through the Chairs Program who are nominated by the partnering federal granting 


agencies. 


Nominations are reviewed by peers against two criteria related to the objectives of 
the Program: a) quality of the nominee and the proposed research program, and 
b) integration with the university's Strategic Research Plan. Three experts from the 
College of Reviewers review each nomination. If the reviewers concur in their 
assessments, a recommendation to fund or not to fund is made to the Program Steering 
Committee. If consensus cannot be reached among the reviewers, the nomination is sent 
to the Interdisciplinary Adjudication Committee for further review. This Committee in turn 
makes its recommendation to the Program’s Steering Committee. 


In terms of feedback following a decision, the CRC Secretariat provides each 
university with copies of the Reviewer Report Forms that are completed during the review 
of each nomination. In those cases where nominations are referred to the Interdisciplinary 
Adjudication Committee, the university also receives a summary of the committee’s 
recommendation. There is no appeal mechanism: 


… [I]f the nomination is rejected, both the university and the nominee can review the 
assessments that were made and may decide to submit the application again, at 
which point we will send the file to three new experts in order to have a fresh look on 
this second review. [René Durocher, Canada Research Chairs Program, 39:16:00] 


Evaluation of the Program 


The first nominations for Canada Research Chairs were received in September 
2000 and the first recipients of Chairs were announced in December 2000. By February 
2002, of 618 nominations received, 554 nominations were approved (90% success rate). 
Of those Chairs approved, 532 had been accepted, with most of them (471) going to 
researchers already in Canadian universities. ® 


In terms of an evaluation of the Program, a scheduled review of the operation and 
structure of the Program is underway; the objective of this review is to identify any 
improvements that could be made to the Program. Any changes to the Program resulting 
from this review will be implemented in September 2002. A comprehensive evaluation will 
be performed in the fifth year of operation. 


'® Statistics supplied by the Canada Research Chairs Secretariat. 
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CHAPTER TWO: SMALL AND REGIONAL 
UNIVERSITIES AND THE ALLOCATION OF 
FEDERAL RESEARCH FUNDS 


| In its June 2001 report, the Committee expressed some concerns about the 
nationwide distribution of federal research grant funds. It was particularly troubled by the 
relatively weak research capacity of small universities and the subsequent concentration 
of federal research funds in a few large institutions. Many researchers from small 
universities claim that they have more difficultly in obtaining research grants from the 
federal granting agencies than do researchers at larger universities. Additionally, certain 
regions of the country (e.g., Atlantic Canada and some parts of western Canada), in part 
because of the higher proportion of small institutions there than in other parts of the 
country, receive relatively low levels of federal research grant funding. The Committee 
recognized that the apparent lower success rates and/or funding levels of researchers 
from small universities were not because these researchers are of inferior quality to 
researchers at larger universities. Instead, barriers such as high teaching loads, small 
graduate programs, inadequate infrastructure, and in many cases, low levels of regional, 
value-added industrial activity probably have a greater impact on the research capacity of 
small institutions. 


The Committee wished to explore further the reasons why researchers from small 
institutions are not faring as well as their large university counterparts under the present 
system of allocating research funds, and whether there are any inherent biases against 
researchers from small institutions in the decision-making processes of the federal 


granting agencies. 


Funding of Researchers at Small Universities 


During the course of its hearings, the Committee heard from several witnesses 
who suggested that researchers from small and regional universities face special 
difficulties in securing funds from the federal granting agencies to conduct their research. 
The granting agencies did not present any data on success rates or funding levels of 
researchers at small (or regional) institutions as compared to their large university 
counterparts, although suggestions were made that the agencies are beginning to 


examine the issue: 


[W]e are aware that small and some medium-sized universities in some regions of 
Canada aren't succeeding as well as they would like. NSERC senior management 
takes this very seriously, and last year we visited 16 universities in the Atlantic and 
prairie provinces to investigate the issue a bit further. We found that there are 
indeed barriers in some universities to research productivity, which can in turn 
significantly affect their ability to compete in an excellence-based peer review 
system. [Elizabeth Boston, NSERC, 39:15:35] 
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The Committee compiled data provided by the granting agencies (for their major 
research grants programs”) and the Association of Universities and Colleges of Canada 
(numbers of faculty at each institution) to construct its own figures on the allocation of 
funds by size of institution (Fig. 1) and by region (Fig. 2) for the three agencies. These 
figures provide an overview only of funding patterns since they represent a single 
competition year (2001) for all three agencies combined, and institution size was 
estimated by counting the number of full-time faculty (not all of whom are active 
researchers) at each university.” Nevertheless, the data do provide a representative 
snapshot of recent funding patterns at the three agencies. Since there are differences 
among agencies in funding levels and patterns, it is important to also examine the 
breakdown of funding patterns by agency (see Appendix 2). The data support the 
anecdotal evidence that researchers at small universities tend to have less success in 
granting agency competitions (Fig. 1a and Appendix 2), and receive smaller grants, on 
average (Fig. 1b and Appendix 2), than their large university counterparts. Consequently, 
funds are concentrated in a few, large institutions (Fig. 1c). Differences in success rates 
among the institutions classified by size are most prominent for SSHRC and CIHR 
(Appendix 2), and are not as noticeable when data for all three agencies are pooled (Fig. 
1a). 


In terms of the regional distribution of funds, researchers from Atlantic Canada 
tend to have slightly lower success rates and smaller grant sizes, on average, than 
researchers from other regions, which is correlated with the relatively high proportion of 
small institutions in Atlantic Canada (see Appendix 2d). Furthermore, of the 16 Faculties 
of Medicine in Canada, only two are located at institutions (Dalhousie University and 
Memorial University) in Atlantic Canada; a correspondingly small proportion of CIHR 
research grants goes to fund medical researchers at these two facilities (see Appendix 
2c). The regional distribution of federal research funds also mirrors the pattern of 
investment by the provinces and/or the private sector in R&D; Ontario and Quebec have 
the largest provincial GERD,”" and receive the largest proportion of research funds from 
the federal granting agencies. 


Research Grants (now Discovery Grants) Program at NSERC, Standard Research Grants Program at SSHRC, 


and Operating Grants Program at CIHR. 


Size of institution: Small = 0-499 faculty members: medium = 500-999 faculty members; and large = greater 


than 1000 faculty members; data on faculty numbers are from 1999 (most recent figures available) and were 
provided by the Association of Universities and Colleges of Canada. See Appendix 2d for list of universities 
grouped by size. 


21 Statistics Canada, Service Bulletin Science Statistics, Vol. 25, No. 8, November 2001. 
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Fig. 1. Success Rates and Average Annual Grant Size for NSERC, SSHRC and 
CIHR Applicants Combined (Competition Year 20017) 


Fig. 1a. Success Rate by Size of Institution 


Fig. 2a. Success Rate by Region 


(29%) 


60 
50 
Sean 2 
Œ c 
a 30 - FA 
8 o 
o 8 
So 20 3 
7) 7) 
10 
0 £ 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Fig. 1b. Average Grant Size by Size of Fig. 2b. Average Grant Size by Region 
Institution 
60000 - 
60000 - 
= 50000 - 
g 50000 - & 
o o 
& 40000 - 40000 | 
5 E 30000 
© 30000 - £ 1 
©) 1) 
o 
® 20000 - ® 20000 - 
5 © 
2 > 
Z 10000 - < 10000 - 
C= - 0 oo . 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Fig. 1c. Amount Awarded by Size of Fig. 2c. Amount Awarded by Region 
Institution ATLANTIC 
PSE $9.4M 
$15.3M (5%) WEST 
(8%) $55.1M 
QUEBEC (28%) 
$55.9M 
LARGE (28%) 
MEDIUM $125.2M 
$57.8M (63%) 


ONTARIO 
$78.0M 
(39%) 


22 


Raw data supplied by the agencies for their major granting programs. 
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The concentration of government research funds in a few large or research- 
intensive institutions is a phenomenon witnessed in other countries (e.g., the United 
States, the United Kingdom and Australia), and is not unique to the Canadian context. In 
the United Kingdom, some smaller universities are attempting to attract more research 
funding by examining the possibility of merging or forming “strategic alliances” (to share 
equipment and laboratory space) with larger universities. The Committee notes this 
trend, but points out that in a geographically large country like Canada, mergers and the 
sharing of infrastructure would, in most cases, be difficult to implement. 


In terms of the success of researchers from small and regional institutions in 
Canada in obtaining research funds, a representative from SSHRC made the following 
observation: 


We have three categories of recommendations that are made by committees. One 
is “recommended and funded.” The second is, “recommended, but unfortunately 
SSHRC does not have the money.” The third category is, “not recommended at 
all.” That second group, recommended but not funded, is where you find a lot of the 
small universities and a lot of the universities over which there are regional 
concerns. [Ned Ellis, SSHRC, 39:15:50] 


The granting agencies contend that low success rates and/or small grants reflect the 
low overall levels of funding provided to the granting agencies by the federal 
government. 


Roots of the Funding Problem for Researchers at Small Institutions 


Most witnesses reinforced the suggestion made during the Committee’s last study 
on innovation that weak research capacity and lack of resources are the major barriers 
preventing researchers from small institutions in competing effectively for research funds 
with researchers from large universities. When these institutions are located in regions of 
the country with low levels of value-added industrial activity and/or low levels of provincial 
investment in R&D, the problems are exacerbated: 


One of the fundamental problems we face as smaller institutions [in obtaining grant 
funds]... is that when we meet the test of excellence and when we submit 
proposals, whether it’s to granting councils or within other types of programs, the 
level of resources available to support those excellent proposals is often the 
problem, as opposed to built-in biases [in the selection process]. [Bonnie Patterson, 
President, Trent University, 51:09:50] 


... AS smaller institutions, they have less well-developed, if any, research and 
development offices, commercialization facilities, and partnerships with national or 
international institutions or organizations. Their capacity to develop any of these 
aspects of research and innovation may be further limited by the financial resources 


# D. Adam, “Manchester merger to spawn research giant,” Nature, Vol. 416, March 2002, p.114. 
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of the province in which they are located. [Jacquelyn Thayer Scott, Advisory Council 


Most witnesses argued that research capacity at small institutions has to be 
improved in order that researchers from small universities can compete on a “level 
playing field” with their large university counterparts: 


It's not a matter here of a bias in the system, which is fundamentally founded on 
quality and excellence. | think those are values that we absolutely have to preserve. 
The issue is, how do we make sure that different institutions with different capacities 
are able to compete with one another effectively? [Tom Traves, President, 
Dalhousie University, 51:09:50] 


Increasing Research Capacity at Small Institutions 


The Committee heard several suggestions as to how research capacity can be 
increased, which would (according to the witnesses) correspondingly increase the 
success rates of researchers in securing research funds at small and regional 
universities. These suggestions included paying the indirect costs of federally funded 
research, providing programs through the federal granting agencies (or other bodies) that 
are targeted to building research capacity in smail or regional universities, ensuring that 
small and regional universities find their research “niches,” increasing funding levels to 
SSHRC, and increasing overall federal funding levels to the federal granting agencies. 


Payment of Indirect Costs 


University representatives argued that the most important first step in “levelling the 
playing field” between small and large institutions would be to establish a permanent 
program to support the payment to universities of the “indirect costs” of federally funded 
research. Indirect costs are those involved with providing the services and physical 
environment necessary for the conduct of research but that are not associated directly 
with a specific project (e.g., expenses associated with providing lighting, central 
computing services, library and research archiving, ; research administration, and 
technology transfer and commercialization offices). Direct costs include salaries of 
research associates, equipment and supplies, travel and subsistence, and 
communication of research results. The salaries of principal investigators are considered 
to be included in the operating grants provided to the universities by the provinces. 


est that the indirect costs of research are increasing as more 
h and researchers are funded. In Canada, the indirect costs 
funded research are paid from university operating budgets; the 
direct costs of research. In the United States, 
her's grant (negotiable with the U.S. granting 


Universities sugg 
university-based researc 
associated with federally 
federal government pays only for the 
universities claim a large portion of a researc 
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councils on a university-by-university basis) to cover indirect costs. Canadian universities 
and university associations have been pressing the federal government to establish a 
permanent program to support the payment of indirect costs: 


As someone from a medium-sized university with a very active research 
environment, | experience first-hand the dilemma that universities face in trying to 
deal with increased enrolment on the one hand, and the increasing complexity of 
research support programs, accountability, or regulatory requirements for research 
with human subjects, animals, and biohazards on the other. A major step forward 
would be for the federal government to reimburse universities for the indirect costs 
they incur in conducting research funded by the granting agencies. [Wayne Marsh, 
President, Canadian Association of University Research Administrators, 43:09:10] 


The Advisory Council on Science and Technology (ACST) has proposed that 
funding for indirect costs should be provided to the universities by the federal 
government. It suggested that each university eventually receive 45 percent of the level of 
funding provided to the university by the three federal granting agencies (based on a 
rolling three-year average of funds received). Under the proposal, smaller institutions that 
receive lower amounts of funding from the granting agencies would receive a larger 
percentage (up to 95%) of these amounts to account for an inability to achieve inherent 
economies of scale in research activities. This increased percentage for small universities 
should directly reduce the implicit cross-subsidy obtained from teaching activities and 
indirectly increase their research capacity which should, in turn, help to leverage more 
funds from the granting agencies for the direct costs of research. At the time the ACST 
released its report (September 2000), it estimated that the final cost of delivering the 
program would be in the range of $450 million annually. 


This Committee, in its June 2001 report, recommended that the government, in 
discussion with the provinces, address the issue of payment of the indirect costs of 
federally supported research. In its December 2001 budget, the federal government 
announced a one-time $200 million payment, with a promise of ongoing support, to help 
support indirect costs at universities and research hospitals: 


Looking ahead, the Government will work with the university community on ways to 
provide ongoing support for indirect research costs that is predictable, affordable 
and incremental to existing support. [The Budget Plan 2001, Department of Finance, 
December 2001] 
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The federal government made its first announcements about the distribution of the 
one-time payment at the end of February 2002. The Canada Research Chairs 
Secretariat IS administering the monies. The formula used to calculate the size of the 
indirect costs grants resembles a “reverse income tax” model, where the first 
$100,000 of research funding from the three federal granting agencies is supported at a 
level of 80% for all universities. The next $900,000 is supported at a rate of 50%, the 
following $6 million at 40%, and the remainder at slightly less than 20%. The 
government's Innovation Strategy” released in February 2002 indicates that supporting 
the indirect costs of federally supported research is one of its priorities for improving the 
university research environment and Canada’s knowledge performance * The 
Committee deems that the permanent support of the indirect costs of research by the 
federal government as being very important. It believes that the formula used to 
calculate the one-time payments is appropriate since the differential costs between 
small and large universities are recognized, but encourages the government to look at 
other models (e.g., that of ACST) when designing any permanent program. The 
Committee thus recommends: 


RECOMMENDATION 1 


That the Government of Canada, in consultation with the provinces 
and territories, instate a permanent program for the support of the 
indirect costs of federally funded research in its next budget, and that 
the formula used to calculate the grant to each institution take into 
account the differential costs between large and small institutions. 


Programs Directed to Small and Regional Institutions 


Some witnesses appearing before the Committee suggested that more programs 
to build research capacity that are targeted to small and regional universities are 
necessary. Building research capacity should lead to an increase in the success rates 
and funding levels of researchers from small universities in national, open competitions 


for research funds: 


More must ... be done to foster and strengthen innovation capacity in smaller 
Canadian universities so that they can continue to strategically harness the 
innovation potential of the communities in which they are located. In advancing a 
new federal capacity building initiative, we acknowledge the need to recognize and 


iversity that has received research grants from at least one of the three 
grant for indirect costs, subject to some restrictions that are listed on the 
the Canada Research Chairs Secretariat’s Web site: 


24 Any degree-granting Canadian un 
granting agencies may receive a 
Indirect Costs section of 
http://www.chairs.gc.ca/english/indirectcosts/qa. html 

ada’s Innovation Strategy can be accessed electronically on the Innovation Strategy 

tionstrategy.gc.ca/cmb/innovation.nsf/pages/index 


g in People, Knowledge and Opportunity, 
f/VRTF/PDE/S$file/achieving. 


25 The two papers forming Can 
web site at http://www.innova 


26 Achieving Excellence: Investin 


htto://www.innovationstrategy. c.ca/emb/innovation.ns df 
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build on the demonstrated flexibility of existing federal programs. The research 
capacity building initiative we propose also recognizes excellence as the 
fundamental criterion of funding allocation. [Robert Giroux, President and Chief 
Executive Officer, Association of Universities and Colleges of Canada, 51:09:10] 


There are already some programs in place at CIHR and SSHRC that are 
specifically intended to assist small universities and/or certain regions of the country build 
research capacity. A description of these programs follows. 


CIHR 


CIHR has an obligation not only to support excellent health research, but also to 
ensure that there is robust health research capacity in all regions of Canada. The 
peer review system does not generally consider regional distribution as one of its 
criteria for evaluation, and we have developed other processes to try to ensure that 
there is good regional distribution of CIHR funds ... [Mark Bisby, CIHR, 39:15:35] 


In 1996, the former Medical Research Council of Canada began a program in 
response to a decline in funding to health researchers in Saskatchewan, Nova Scotia, 
Newfoundland and Manitoba. The Regional Partnerships Program (RPP) was expanded 
in June 1999 to include Prince Edward Island and New Brunswick. The Canadian 
Institutes of Health Research continues to offer this Program. In addition to funding health 
research, the RPP supports a local strategic planning process to establish research 
priorities and partnerships, emphasizing the recruitment and retention of promising and/or 
excellent researchers, building on local strengths and priority interests of the institutions. 


Under the Program, research funding and personnel support applications that are 
judged to be of high scientific merit through peer review, but are below the funding 
capacity of CIHR’s base budget in regular competition, are eligible to receive funding if 
there is a partner to co-fund the proposal. The annual maximum CIHR co-funding is 
$1 million for each of the original four provinces (Manitoba, Newfoundland, Nova Scotia 
and Saskatchewan) on a 1:1 partner-funding ratio. For the Program’s newest provinces 
(Prince Edward Island and New Brunswick), funding has been set at a level of 
$200,000 per annum at a 1:1 partner funding ratio. CIHR’s current commitment to the 
Program is $4.4 million per annum. The Program has had positive effects in increasing 
the research capacity of institutions in the participating provinces in the area of health 
sciences: 


[T]he Medical Research Council a number of years ago instituted a program, 
which... set aside a designated amount of what was essentially seed 
money — small grants — for a designated number of smaller institutions that 
allowed us to invest in our good ideas. So when we eventually did go forward to 
compete with scholars at much bigger and better funded universities, we had the 
same pre-investment in our project and had made the same progress as they had. 
So we were able to compete on a level playing field. Significantly our research grant 
results went up substantially as a result of that kind of support. [Tom Traves, 
President, Dalhousie University, 51:09:50] 
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- CIHR also established the Institutional Establishment and Development Grants 
Program, which is intended to increase capacity in health research at Canadian 
institutions. The Development Grants portion of the Program is intended to permit certain 
underdeveloped” institutions” to undertake activities that will increase their 
competitiveness in applications for CIHR funding. The Program provides a one-time grant 
of up to $100,000 per institution (expiring on March 31, 2002). At present, this Program is 
a one-time funding opportunity for institutions, pending submission of final reports by the 
institutions and an evaluation of the Program’s outcomes in 2003. The rationale behind 
establishing Development Grants stems from the fact that most institutions have identified 
health research as a priority for development in their Canada Research Chairs Program 
(CRCP) Strategic plans, yet investigators at many institutions currently receive very little 
funding from CIHR. Since CRCP allocations are based on past funding received from the 
three federal granting agencies, such institutions receive few or no CIHR-CRCP 
allotments. The purpose of development grants is to assist underdeveloped institutions to 
build and mobilize their health research capacity, which should, in turn, lead to CRCP 
allocations in health research and increased research growth. 


SSHRC 


The Aid to Small Universities Program offered by SSHRC is intended to help small 
universities develop and strengthen focused research capacity in the social sciences and 
humanities. Small institutional grants are awarded on a competitive basis for a three-year 
period and support such activities as start-up costs/partial funding of research centres; 
stipends to doctoral students; organization of colloquia or symposia; and seed funding for 
collaborative research or the development of partnerships. The maximum value of a grant 


is $30,000 per year for three years. 


The SSHRC Institutional Grants Program, which is open to all eligible Canadian 
postsecondary institutions and helps to fund small-scale research activities by faculty in 
the social sciences and humanities, also assists universities in strengthening research 
capacity. Under the Program, SSHRC provides block grants (minimum of $5,000) for 
three-year periods to each eligible institution. The institutions use these funds to award, 
through their own competitive process, small individual grants to support research 
activities such as short-term research projects, research-related conference travel and 


organizing small seminars. 


Other SSHRC policies indirectly ensure that funds in certain programs are shared 
among institutions across the country: 


1 ADM ee 
27 Those institutions that have received less than 1% 
and are eligible to receive Canada Research Chairs. 


of the CIHR grants and awards budget in fiscal year 2000-01, 


In certain programs sometimes we will introduce particular clauses, in the 
Community-University Research Alliance Program,” 8 for instance, which was, of 
course, community-based and had a huge demand. What was nice about that 
program was that we specified that institutions could only get one 
Community-University Research Alliance, which meant that it was much more 
widely spread across the country. [Ned Ellis, SSHRC, 39:15:45] 


NSERC 


Although NSERC has had programs directed to small universities in the past, no 
such programs exist today because of budget constraints. The agency is in favour of 
targeting resources to build the research capacity of small and regional universities that 
would allow these institutions to be more successful in national competitions: 


[The universities] need targeted resources to bring their research facilities and 
capacity to a high level, to allow them to compete on a level playing field, and a 
relatively modest program to provide flexible infrastructure support could go a long 
way to achieving this goal. This advice has been made available to Industry Canada 
and the federal regional economic development agencies. [Elizabeth Boston, 
NSERC, 39:15:35] 


Focusing on Research Strengths 


The importance of small institutions identifying their areas of research strength and 
focusing their efforts in those areas was emphasized by several witnesses. Strategically 
focusing research efforts in an area of speciality allows a small university to build 
research capacity and productivity and to excel, both nationally and internationally, in that 
area: 


In terms of capacity building for research, many of us find it is critically important to 
our success to be very niche-focused. We need to develop critical mass in particular 
areas. Some of the programs the federal government has put in place in fact are 
now allowing us to establish excellence at a critical mass level. [Bonnie Patterson, 
President, Trent University, Association of Universities and Colleges of Canada, 
51:09:30] 


Another witness argued, however, that some small institutions because of their 
mandates and/or types of industry in the region might have difficulty expanding into 
other areas of research: 


Behe Community-University Research Alliance Program was a pilot program begun in 1999 that funded research, 


training and knowledge-sharing partnerships between universities and communities. The pilot program was 
heavily subscribed, but SSHRC suspended the program for the 2001-02 competition because of lack of funds. 
SSHRC’s Council decided in March 2002 to launch a new competition (letters of intent stage only, pending an 
examination of the availability of funds for a full scale program) for 2002-03. 
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[RJesearch activities in smaller institutions with subregional development mandates 
are generally less well developed ... Typically, they begin with involvement in some 
form of sectoral economic activity within their subregion that may support a very 
limited number of mid-sized or larger employers ... It may be difficult for such 
institutions to move into other niche areas that will focus on assisting SMEs or 
helping to diversify the base for their local economy. [Jacquelyn Thayer Scott, 
Advisory Council on Science and Technology and President and Vice-Chancellor, 
University College of Cape Breton, 55:09:45] 


Despite this caution, the Committee supports the idea that small institutions should 
Strategically focus their research efforts in one area rather than trying to engage in 
“across the board” research activities. A suggestion was made that as part of any 
program intended to build research capacity at small or regional universities, the 
universities should be required to identify their areas of research strength, and to request 
funds to help promote research capacity in that area: 


This is why we suggested a program that would target smaller universities. [The 
universities] would submit a plan to strengthen their capacity in one area and would 
outline their needs. This program could be available to finance the capacity-building 
side of university research. [Robert Giroux, AUCC, 51:09:40] 


Within these programs, some witnesses made the point that excellence should still 
remain the primary selection criterion for the allocation of funds. 


The Committee was encouraged to learn that some programs to help certain small 
and regional institutions are already in place at the granting agencies. These types of 
programs are essential in assisting these institutions build research capacity in areas of 
strength, and ultimately in helping their researchers to obtain a larger proportion of 
research funds, through open competition, from the granting agencies. The Committee 


therefore recommends: 


RECOMMENDATION 2 


That the Government of Canada, in consultation with the provinces 
and territories, set up targeted programs through the granting 
agencies to assist “underdeveloped” small and regional institutions 
strengthen their research capacity in key areas, and that 
appropriations be made to the agencies to fund these programs. 


Increased Funding for Research in the Social Sciences 


The Social Sciences and Humanities Research Council of Canada currently 
receives the lowest level of funding of the three granting agencies (projected base budget 
of $157 million for fiscal year 2002-03 as compared to $559 million for NSERC and 
$562 million for CIHR). The agency argues that it is under-funded since it represents 
55% of all university researchers, but receives only about 13% of the federal investment 
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in university-based research. The Committee notes this difference, but suggests that part 
of this funding gap may be explained by the lower costs, on average, of research in the 
social sciences as compared to much of the research in the natural sciences, engineering 
and health sciences. Differences in amounts of federal funding among different areas of 
research may also reflect differences in the socio-economic impact of the research. The 
Committee encourages SSHRC to place more effort into promoting to the public the 
impacts of SSHRC supported research on Canadian society. 


A few witnesses appearing before the Committee suggested that since many 
smaller institutions have their research foci in the social sciences and humanities, 
providing adequate levels of funding in these areas is critical for ensuring that researchers 
in small institutions receive appropriate levels of federal funding for their work: 


The second factor for many of us [at small universities] is that we are largely 
focused on the social sciences and humanities. At Trent, for example, our focus is 
approximately 60% in that area. Therefore, access to resources through such 
granting councils as SSHRC becomes fundamental to the ability of our universities 
to compete, not just regionally and nationally but internationally. [Bonnie Patterson, 
President, Trent University, 51:09:30] 


... [W]e feel the time has come to address the historic underfunding of the 
SSHRC ... Again, that Council plays a particularly important role at many smaller 
institutions in the country ... [Wayne Marsh, President, Canadian Association of 
University Research Administrators, 43:09:10] 


The Committee is concerned about the level of funding to SSHRC and especially 
its impact on researchers at smaller and regional universities. It notes that relatively low 
levels of funding at SSHRC have resulted in the suspension of some programs, and an 
inability for SSHRC to participate in certain initiatives. For example, SSHRC has been 
unable to provide any monies to follow up on recommendations made to NSERC and 
SSHRC by the Task Force on Northern Research to rebuild Canadian northern research 
capacity, whereas NSERC has been able to channel some of its funds to at least partially 
implement the recommendations.®” To address the issue of differences in funding levels 
among agencies, the Committee recommends: 


RECOMMENDATION 3 


That the Government of Canada increase the level of funding to 
SSHRC, especially given the impact of this funding on researchers at 
small institutions. 


*° The Task Force’s recommendations can be accessed electronically in its report From Crisis to Opportunity: 


Rebuilding Canada’s Role in Northern Research, ftp://ftp.nserc.ca/pub/nserc_pdf/nor/crisis.pdf 


22 


The Committee believes that officials at Industry Canada, who are familiar with the 
agency s mission and the outcomes and impacts of its programs, are in the best 
position to decide on the size of the increase to SSHRC’s budget. 


Increased Funding to All Three Federal Granting Agencies 


Some witnesses appearing before the Committee argued that low success rates 
and funding levels to researchers at small institutions reflect inadequate levels of funding 
to the three federal granting agencies, not flaws in the peer review system itself: 


| think the real problem is not redistributing the dollars and not putting the finger on 
the problems of process as being the cause of the other problems. | think the 
problem is that we don’t spend enough on R&D in the country as a whole. [Michael 
Piva, Canadian Association of University Teachers, 43:10:00] 


[T]he fundamental, underlying issue is the overall lack of funding in the granting 
councils in particular. You will therefore get situations in which people are writing 
good proposals time after time, but they’re not being funded. That’s because in the 
judgment of the people doing the reviews, is not that it's not meritorious research, it 
just comes a little too far down on the scale and the funding doesn’t come down far 
enough to meet it. [Wayne Marsh, President, Canadian Association of University 
Research Administrators, 43:10:10] 


The Committee appreciates that higher levels of funding are required for 
university-based research in Canada, and notes that this need is also recognized in the 
government’s /nnovation Strategy.” The Committee believes that larger annual increases 
to the granting agencies (i.e., larger than the 7% increases awarded to SSHRC and 
NSERC in the December 2001 budget) are necessary to help meet the federal 
government’s target of doubling its annual investment in R&D by 2010. The Committee 


recommends: 


RECOMMENDATION 4 


That the Government of Canada immediately increase funding to all 
three granting agencies so that success rates and/or grant levels for 


excellent research can be increased. 


The Committee encourages the government to consider a doubling of agency funding 
levels by 2010, and ensure that annual increases to the agencies reflect this long-term 


goal. 


The Committee recognizes that the recommendations made in this section of the 


report will lead to results only if the institutions and regions involved also work towards the 


PRE LL eee RE REA 


à (evi : sting 
Achieving Excellence: Inve 
htto://www.innovationstrategy. c.ca/cmb/innova 


in People, Knowledge and Opportunity, | 
tion.nsf/VRTF/PDE/Sfile/achieving. 


df, p. 52. 
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goals of improving regional and institutional research capacity. Issues such as the levels 
of provincial support for research (for both direct and indirect costs) and for universities, 
as well as institutional support for researchers, have an important impact on research 
capacity and productivity, independent of federal efforts in this area. Furthermore, the 
Committee notes that stronger collaboration between the provinces, territories and the 
federal government on such research funding problems would make it easier to find 
solutions. Similarly, increased collaboration on other issues related to research, 
development and innovation would improve the innovation performance of Canada, the 
provinces and the territories relative to their international competitors. 


Potential Bias Against Small Institutions in Decision-making Processes 


An argument was made that the present peer review system is biased towards 
supporting researchers at established, large institutions since the selection criteria focus 
on such factors as the quality of an applicant’s publication record and the numbers of 
highly qualified personnel trained, which are usually stronger for researchers at large 
universities. One witness argued that other evaluation standards need to be applied to 
researchers at small institutions, which are often newer and located in rural or remote 
regions of the country. The witness indicated that there are different and equal standards 
of excellence that are more appropriate to different contexts, and that separate funding 
programs, targeted to researchers at small institutions, are needed at the granting 
agencies: 


That really is what the ACST has been saying in at least two of its reports, that 
we’ve been doing very good things related to strengthening the good things that 
already exist in Canada’s well-developed universities in metropolitan regions, but 
now we need to have separate programs that are equally excellent, but with different 
standards for the excellence appropriate to other communities that are at a different 
developmental stage. [Jacquelyn Thayer Scott, Advisory Council on Science and 
Technology and President and Vice-Chancellor, University College of Cape Breton, 
55:09:45] 


However, other witnesses argued that although many researchers from small 
universities do face problems in securing funds from the granting agencies, they did not 
want the selection criteria and peer-review processes altered. These witnesses 
suggested that the present excellence-based, competitive decision-making process is 
fundamental to the efficient allocation of research funds. Instead, they recommended that 
the issue of weak research capacity be addressed in order that researchers from small 
universities can compete in open competition with their large university counterparts: 


... Peer review isn't perfect, but it's by far the best imaginable system. | stress that 
its the only one that can make us internationally competitive ... To make that 
reversal of our relative decline [in productivity] and to be economically competitive, 
you have to allow for a competitive system in the allocation of the grants. Yes, do 
the capacity building. Yes, do the special programs we've all been talking about, but 
my goodness, peer review is the heart of what translates the dollars into excellence. 
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Give up On peer review and you give up on the excellence. [Paul Davenport, 
President, University of Western Ontario, 51:10:40] 


Lob Committee recognizes the importance of maintaining a competitive system for the 
allocation of funds, but it encourages the granting agencies to ensure that the special 


situations of small and regional universities is taken into account in all aspects of the 
review process. 


At this stage, the Committee believes that it is important to address the issues of 
the payment of indirect costs, disparities in research capacity between small and large 
institutions, and low overall funding levels to the granting agencies. The Committee will 
monitor the success rates and funding levels of researchers at small and regional 
institutions to see whether improvements in the aforementioned areas are correlated to 
improved results for these researchers in granting agency competitions. If major 
improvements are not witnessed, other factors (including, possibly, the peer-review 
process itself) are likely responsible for the relatively poor performance of researchers at 
these institutions. The Committee encourages the federal granting agencies to review 
their funding mechanisms to ensure that researchers from small and regional institutions 
participate actively in decision-making processes, and that, as far as possible, the review 
process take into account the difficulties faced by researchers at these institutions. The 
Committee will continue to monitor the situation and determine whether further 
investigation and changes are required. 


The Canada Research Chairs Program and Small Universities 


The Committee’s concerns about the allocation of Chairs under the Canada 
Research Chairs Program were two-fold: (1) the formula used to calculate the allocation 
of Chairs to universities is based on the past performance of universities in obtaining 
federal research grant funds which might reinforce the existing disparity in research 
capacity across the country; and (2) the Program might lead to the “poaching” by large 
universities of the best faculty from smaller universities thus augmenting even further the 
disparity in research capacity across the country. Some universities and university 
associations had the same concerns when the Program was launched in 2000. 


In general, witnesses appearing before the Committee stated that their initial 
concerns about the Program have largely been erased. With respect to the allocation 
formula, witnesses suggested that the Program provides some flexibility in how the Chairs 


can be used by smaller institutions: 


all, one of our experiences with the Program is that where there can be 
the happier and easier it is in some particular disciplines to 
of our highest-end researchers approaching 
se a senior-awarded Chair in two particular 
helpful in our case. | would speak to the 


First of | 
flexibility with the Chairs, 
address issues. As we see some 
retirement, the ability, for example, to u 
areas as junior Chairs has been very 
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structure of the Program being fine as we're moving forward. [Bonnie Patterson, 
President, Trent University, 51:10:00] 


As for the issue of raiding, the Chair of the Association of Atlantic Universities made the 
following comments: 


My region was certainly one that was very anxious about this. As | indicated, we 
have many small institutions. There was a great anxiety at the start of the Program 
that we would essentially be raided — that the best people at our institutions would 
disappear into the better-funded, larger institutions across the country. That has not 
happened. At my university we’ve lost one person. But at the same time we've 
attracted people from other institutions. If you just see this in the context of the 
normal movement by people in terms of their careers, it fundamentally has not 
proven to be the kind of anxiety we worried about. [Tom Traves, President, 
Dalhousie University, 51:09:55] 


The Committee is still concerned about the allocation of Chairs to small universities and 
encourages the Canada Research Chairs Secretariat to pay special attention to this 
issue in present and future reviews of the Program. 
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CHAPTER THREE: EXCELLENCE, RELEVANCE, 


AND STRATEGIC RESEARCH 


The issue of funding priorities for Canada, and their relationship to granting agency 
programs and selection criteria, came up on a number of occasions during the 
Committee’s hearings. Most of the witnesses appearing before the Committee stressed 
that excellence should be the fundamental selection criterion for any peer-review process 
for the allocation of federal research funds. However, for certain types of research 
programs, the short-term socio-economic relevance of the proposed research, sometimes 
in specified target areas, is also very important. The question of what proportion of 
granting agency funds should be dedicated to supporting research in particular target 
areas is a contentious issue. Some critics argue that certain research areas of 
socio-economic importance to Canada are not receiving adequate funding and that more 
funds should be channelled to those areas. Other commentators argue that there is 
already an adequate proportion of granting agency funds directed towards supporting 
research in target areas, and more funds should not be directed there at the expense of 
funding for other types of research. The issue of how much funding should be directed to 
different research areas is clouded, according to come critics, by a lack of structured 
government policy in this area. These questions are explored in this chapter. 


Relevance and “Strategic” Granting Programs 


In addition to the excellence of the research proposal and investigators, and the 
need for funds, other factors are considered for certain programs when deciding on the 
allocation of research funds. For example, some programs at the federal agencies 
examine the industrial relevance of the proposed research and whether the project is 
likely to generate economic spin-offs for the Canadian economy within a reasonable 
period of time. Other programs focus on the socio-economic relevance of the proposed 
research in particular target areas of national importance. The objective of these 
“strategic” programs is to promote research and training in target areas with the ultimate 
goal of transferring knowledge or technology expertise to Canadian-based organizations 
that have a capacity to use the results; all three granting agencies offer some form of 
dedicated, strategic research programs. The research is usually conducted in conjunction 


with government or industry-based partners. 


In fiscal year 2000-01, these strategic programs accounted for approximately 7% 
of expenditures at NSERC (spending on other non-targeted research with societal or 
industrial relevance conducted in partnership with vile) and SES UE agencies 
accounted for a further 9% of NSERC expenditures), 15 7% at SSHRC, and 20% at CIHR 
(Networks of Centres of Excellence not included in the calculations for the three 
agencies). Expenditures on strategic research at CIHR is expected to grow to between 
30% and 40% of the total budget as the agency’s institutes, which began operating in 
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February 2001, expand. Other research in target areas of national importance to Canada 
is funded through non-targeted research programs, so the actual expenditures on such 
research by each agency are higher than the amounts reported here. The proportion of 
research in target areas funded by the granting agencies has increased over the past few 
years: 


Look at the agencies’ budgets ... There has been an evolution over the course of 
the last 20 years ... They have all, without exception, followed this trend. Today, 
between 20 and 30% of the agencies’ budgets, agencies which up until very recently 
only funded non-directed, non-strategic fundamental research, [are] already devoted 
to such programs. [Benoit Godin, Observatoire des sciences et des technologies, 
66:09:55] 


The percentage of expenditures at SSHRC on targeted research programs will be 
higher for 2002-03 because of the funds ($100 million over five years) that the 
government has invested in the /nitiative on the New Economy (INE), which are managed 
by SSHRC. The main goal of this initiative is to help Canada and Canadians adapt to and 
benefit from the “New Economy.” The INE will focus on four major areas of research (that 
were selected by the federal government in consultation with a variety of stakeholders): 
general new economy issues, management and entrepreneurship, education and lifelong 
learning. A major departure in the evaluation process compared to other SSHRC 
programs is that before INE applications proceed to the adjudication stage, an 
interdisciplinary committee of academics and non-academics, with experience in the 
areas relevant to the four INE themes, screens the applications for relevance to INE 
objectives. Only applications that pass this “relevancy test” are sent on for traditional peer 
review. 


For strategic research programs, the granting agencies all have mechanisms in 
place to choose and modify target areas. The methods used and the time periods 
between reviews of the areas vary among the agencies. At NSERC, the Strategic 
Projects Program supports research in target and emerging areas of national importance. 
The Program operates on a five-year cycle; the Program’s most recent evaluation of 
target areas was in 2000. The target area evaluation begins with a study of the existing 
national and international material on priority research areas identified by the private and 
public sectors. The results of this study are then validated with leaders in the academic, 
industrial and government sectors. In 2000, the strategic areas selected were: 
biosciences, environment and sustainable development, information and communications 
technologies, value-added products and processes, and “new directions.” At NSERC, the 
allocation of funds to various disciplines is also examined and modified every four years 
in its main Discovery Grants Program, which funds basic and applied research. Funds are 
shifted among disciplines during the “Reallocations Exercise” to ensure that science and 
engineering that is important for Canada receives appropriate levels of funding under the 
system. 


At SSHRC, different methods have been used over the past 10 years to select 
Strategic research areas. The selection is usually made following broad consultation with 
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academics, university administrators, government representatives, members of 
non-governmental organizations and other stakeholders. An “environmental scan” 
highlighting knowledge gaps in the social sciences and humanities is produced following 
the consultation process and is presented to SSHRC’s Governing Council. A short list of 
theme areas is drawn up, discussed with the community, and then the Council makes 
final decisions on theme areas to be selected. Reviews of the theme areas are made 
every three to five years to ensure that the areas selected are still relevant. The most 
recent set of strategic themes, selected in March 2002, are: culture, citizenship and 
identity (including issues involving peace and security); environment and sustainable 
development; texts, visuals and technology; and Aboriginal peoples. 


At CIHR, the individual Institutes decide on thematic initiatives based on a rolling 
three- to five-year strategic “outlook.” The outlooks are developed by the Scientific 
Director of each Institute, working in partnership with each Institute’s multi-stakeholder 
Advisory Board. The CIHR Governing Council approves these broad outlooks, and from 
them the Institutes develop specific plans for research investments in the upcoming fiscal 
year. These plans are presented to the Research Planning and Priorities Committee 
(RPPC), which is made up of CIHR’s President, Vice-President (research portfolio) and 
the 13 Institute Directors, for appraisal and modification prior to launch. Following the 
publication of CIHR's first overall strategic outlook,” the RPPC is now considering a 
number of proposals for major cross-cutting research priorities which would involve a 
number of Institutes working together, in addition to the priority areas defined by each 
Institute individually. Examples include such themes as rural and northern health, 
gene-environment interactions and disease, and tobacco control. 


Other research in target areas is also supported through the granting agencies. 
The Networks of Centres of Excellence (NCE) Program is a tri-council and Industry 
Canada initiative that was established in 1989 and made a permanent Program in 1997. 
The Program promotes partnerships among universities, industry, government and 
non-governmental organizations. There are currently 22 networks in four areas: health, 
human development and biotechnology, information and communication technology, 
natural resources, and engineering and manufacturing. Its present budget stands at 


$77.4 million per year. 


The identification of research priorities within the NCE Program has been achieved 
in two ways: (1) through a bottom-up approach in which applicants propose a NCE ina 
given area and the applications are judged in open competition; and (2) through a 
top-down approach in which specific research areas of national importance are identified 
in advance and proposals invited to address those areas. The first of these targeted 
competitions was conducted in 1995, the second one In 1999. Target areas are identified 
through consultations with the research community, with industry and with government. 
Prior to each competition, the NCE Steering Committee (composed of the presidents of 
the three federal granting agencies and the Deputy Minister of Industry Canada) decides 


i http:/www.cihr.ca/news/publications/publications/revolution. pal 
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on the need to target areas for new networks and, if the decision is to target, 
recommends targeted areas to the Ministers of Industry and Health. The federal 
government (Cabinet) makes the final decision. 


Research in Interdisciplinary and Emerging Areas 


Much of today’s research is interdisciplinary, crossing discipline boundaries within 
agencies and, in an increasing number of cases, crossing the boundaries of granting 
agency mandates. Other research is in emerging or small areas that does not fit well into 
the traditional, discipline-based structure of the major “open” granting programs at many 
agencies. Some researchers contend that research in interdisciplinary, emerging or small 
fields is disadvantaged under the present system. The Committee heard of such an 
example from a researcher who works in the area of animal health research: 


NSERC has neither a mechanism nor the expertise to review or fund this 
[population] real-world [animal health] research. NSERC’s perspective on the issue 
is that there is no problem, since it doesn't receive proposals for funding of this type 
of research. The reason they receive no proposals is that investigators will not 
submit proposals until there is an appropriate grant selection committee in place. 
[lan Dohoo, Professor, University of PEI, 58:09:40] 


The witness believed that areas not covered by the present committee structure are 
disadvantaged in the competition for funds. He argued that because of limited funds, 
committee members are reluctant to recommend funding for research in new areas at 
the expense of funding for the majority of proposals that are in more traditional areas. 
For animal health research, the witness suggested that NSERC should follow the 
example of CIHR in its approach to funding research: 


So what's the solution? Well, in the recent transformation of the Medical Research 
Council into the Canadian Institutes for Health Research, the need to fund the full 
spectrum of health research from basic laboratory research to real-world or 
population- and clinical-based research was clearly recognized, and that’s been built 
into the four pillars each institute is built upon. Ideally we need the same sort of 
transformation — the same approach to animal health research — at NSERC. [lan 
Dohoo, Professor, University of PEI, 58:09:40] 


NSERC contends that it funds both laboratory-based and population studies in animal 
health, and that a variety of NSERC programs are available to support animal health 
research. Despite this assurance, the Committee encourages NSERC to review its 
mechanisms for funding animal health research (in collaboration with other federal 
departments that work in this area, if necessary) to ensure that this area of research is 
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receiving proper consideration. Additionally, the Committee notes that Canada’s four 
veterinary colleges are facing other problems related to outdated or inadequate 
infrastructure that, if not remedied, could result in the colleges losing their international 
accreditation. Given the importance to Canadians of the research conducted by the 
veterinary colleges in such areas as food safety and animal disease, the Committee 
encourages the federal government, granting agencies, veterinary colleges and 
provinces to work together to quickly resolve these problems. 


For interdisciplinary research and research in emerging areas, the Committee 
heard that the small pool of reviewers available is a problem in terms of evaluating 
proposals in these areas: 


We have an excellent system for judging incoming proposals. If it has flaws, it’s 
because the pool of academic reviewers is too small with the result that there is 
reviewer fatigue. This is particularly true for interdisciplinary panels. The problem is 
often compounded since the individual who may be familiar with your work often has 
to withdraw simply because they are in some way part of your small community with 
the result that the proposal does not receive the detailed review that it should. [J. 
Adam Holbrook, Associate Director, Centre for Policy Research on Science and 
Technology, Simon Fraser University, 66:09:10] 


An example of the “small reviewer pool” problem was provided to the Committee for the 
area of polar research. The witness suggested that polar research is disadvantaged by 
the system, because of the small research community and consequently a lack of 
researchers who can provide unbiased reviews of polar research proposals: 


The small and shrinking community of polar scientists, in many disciplines, results in 
the fact that we know each other very well; we’ve usually co-operated on research 
projects; we’ve frequently co-authored papers with each other; and established 
partnerships in terms of funding with the granting councils, with government 
agencies and other people. And, as a consequence, establishing a peer review 
panel for major research proposals or even proposals within the current granting 
council system is really quite difficult. [Peter Johnson, Chair, Canadian Polar 


Commission, 75:09:15] 


In the long term, the witness argued that the solution to improving the situation for the 
evaluation of polar research proposals is to build up the polar research community (for 
example, by implementing the recommendations of the Task Force on Northern 
Research®’). In the short term, the witness suggested that agencies should rely more 
on international reviewers to evaluate proposals in polar research (the agencies already 
do solicit the help of international reviewers in their review processes). 
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At CIHR, efforts are being made to ensure that the review process reflects its new, 
broader, interdisciplinary mandate: 


The peer review system at CIHR is undergoing a period of evolution. We've 
introduced a dozen new committees in the past year to deal with increased 
application pressure and to accommodate CIHR’s broadened mandate, and we're 
creating about 15 ad hoc committees to review the strategic or thematic 
initiative .... To try to find the best fit between a specific proposal and the team of 
reviewers that evaluates it, we'll be forming clusters of similar peer review 
committees, which will meet at the same time, allowing for a flexible committee 
membership, better tailored to the review of a wide range of proposals. This cluster 
organization should also stimulate a more rapid evolution of committee mandates to 
anticipate, rather than react to, changes in the directions of health research. [Mark 
Bisby, CIHR, 39:15:30] 


For their major research grants programs, both NSERC and SSHRC have 
interdisciplinary committees that evaluate research proposals at the boundary between 
disciplines or involving several disciplines that cannot easily be reviewed by the traditional 
discipline-based committees. The committee members represent a variety of disciplines. 
At NSERC, the interdisciplinary committee often requests reports on proposals from 
members of other grant selection committees, in addition to reports requested from 
external experts, in helping it reach a funding recommendation. 


High-risk Research 


Given that, on average, industry tends to shy away from conducting research with 
a high level of risk, governments have an important role to play in helping to fund this type 
of high-risk, high-payoff research. One witness contended that peer review works well for 
most types of research, including interdisciplinary research, as long as selection 
committees are appropriately structured. However, the witnessed questioned whether 
high-risk research receives proper consideration under peer review. According to the 
witness, the inherently high risk of failure associated with this type of research, the 
scarcity of funds, and the tendency for committees to reach consensus by making 
conservative funding recommendations lead to a bias against this type of research: 


[PJrobably the most serious problem with peer review is how it treats high-risk 
research. Despite the many federal agency pronouncements on the importance of 
supporting high-risk, high-payoff research, in reality there are few incentives and 
motivations for promoting truly high-risk research and there are many disincentives. 
... Use of committees for performing peer review, especially large committees, 
which is characteristic of many of the funding agencies, intrinsically leads to 
conservative judgments. [Ronald N. Kostoff, 88:10:10] 


The witness suggested that “top-down” agency mechanisms may be the only way 
to ensure that this type of research receives appropriate levels of funding: 
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! don't really have an answer to [the problem] other than hiring [agency staff] who 
basically are willing to take these risks and are willing to accept the failures. It 
becomes a very personal issue. | don't see how one can in a sense legislate that 
organizations should be taking risks. [Ronald N. Kostoff, 88:10:35] 


In some foreign funding agencies (e.g., the National Science Foundation in the United 
States), program officers or directors have the authority, in some cases, to allocate a 
small proportion of the budget envelopes that they manage to fund high-risk research or 
research in emerging areas. Furthermore, staff can override the funding 
recommendation of a selection panel if the panel's decision is considered to be overly 
conservative. At the Canadian federal granting agencies, staff overseeing the 
operations of peer review committees does not generally have the authority to make 
independent funding decisions, alter budget envelopes, or unilaterally override the 
funding recommendations of the committees. 


The Committee realizes that the major “open” granting programs at the Canadian 
agencies allow researchers to pursue other, often riskier, lines of research than those 
detailed in their grant applications. The agencies, as part of program evaluations, are 
examining the support of high-risk and interdisciplinary research. Some effort (e.g., at 
CIHR) is being made to provide funds specifically for high-risk research projects. The 
Committee encourages the agencies to continue to monitor and improve the mechanisms 
for the support of high-risk research. 


Increased Collaboration Among the Granting Agencies 


The Committee was generally pleased with the mechanisms in place within each 
of the granting agencies, especially at the CIHR, to assess and modify the distribution of 
research funds within strategic programs. It questions, however, whether research in 
target areas of national importance is receiving adequate support from the overall funds 
available at NSERC and SSHRC, although it realizes that such research is also being 
funded through non-targeted programs at those agencies. Additionally, the Committee is 
concerned about whether support for research in emerging fields and high-risk or 
interdisciplinary research is receiving adequate consideration under the present system. 
The Committee is worried that interdisciplinary research, often in strategically important 
areas, that crosses the boundaries of the three granting agencies’ mandates (other than 
that supported through the NCE Program) is not receiving adequate consideration and 


appropriate levels of funding. 


The Committee notes that the seven research councils in the United Kingdom 
have established “The Cross Council Research Forum,” an informal group of individuals 
whose goal is to improve cross-council operations. One of the topics it has tackled is the 
review of proposais that are at the ‘interface’ of the six grant-awarding research councils. 
The Committee encourages the Canadian granting agencies to review the general 
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operating principles established by the U.K. councils for reviewing such proposals to 
see whether there are any novel principles that might be appropriate in the Canadian 
context. In May 2002, the U.K. government launched Research Councils UK which, 
working with the Office of Science and Technology, will develop new areas of 
collaboration in key scientific fields. It will build on areas where research councils are 
already working together, including the provision of interdisciplinary programs in such 
areas as genomics, e-science, basic technologies, stem cell research and climate 
change. Research Councils UK will also strive to harmonize procedures and provide a 
more efficient service for the councils’ applicants, by, for example, developing a single 
application route for all research councils. 


The Committee believes that the Canadian federal granting agencies could 
co-operate further to support strategic and interdisciplinary research that falls under the 
purview of all three agencies. It also believes that cross-council operations in general 
could be improved. The Committee recommends: 


RECOMMENDATION 5 


That the Government of Canada encourage the granting agencies to 
collaborate further with each other and with non-academic 
stakeholders, including the provinces and territories, when deciding 
on target areas within strategic programs. The agencies should also 
ensure that formalized review mechanisms and adequate levels of 
funding are available to support high-calibre interdisciplinary research 
or research in emerging areas, especially that which crosses the 
boundaries of the agencies’ mandates. The agencies should consider 
establishing a formal mechanism to facilitate collaboration on all 
levels so that their operations are more uniform, allowing for 
“seamless” access by the research community to federal research 
funds. 


Basic and “Curiosity-driven” Research 


Research that is deemed to be of immediate socio-economic importance to 
Canada is usually easier to justify to the government and public in terms of it being “value 
for money” than is basic research. The former, however, often depends heavily on 
advances in areas of basic research that may not have obvious or immediate value in a 
non-scientific arena: 


* http://www.research-councils.ac.uk/researchforum/peerreview.htm 


% The formation of Research Councils UK results from the implementation of a recommendation made in the 


government's Quinquennial Review of the Grant-Awarding Research Councils (2001). 
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we advances in biology and human health tomorrow will come from fundamental 
understandings in physics, in social sciences, in philosophy, etc. [Matthew Spence, 


President and Chief Executive Officer, Alb i 
Lt a 3 erta Heritage Foundation for Medical 


In many cases, the socio-economic value of a particular research program is not 
realized until years down the road when a discovery emanating from that research is 
applied to a specific problem. The majority of federal granting agency funding in 
Canada goes to fund research in areas chosen by the investigators themselves 
(“curiosity-driven” research, which may or may not have immediate industrial or societal 
relevance), and is not channelled into funding research in target areas selected by the 
granting agencies or government. According to some witnesses, most scientists support 
such a system and suggest that it may lead to much more useful research in the longer 
term than directly supporting research that has immediate and obvious socio-economic 
value and/or is in a particular target area: 


The great thing about our Canadian system is that we have enough autonomy 
among our faculty and universities that we can have this interest-driven research 
that often is far more valuable in the long run than research that has an immediate 
commercial application. [Paul Davenport, President, University of Western Ontario, 
51:10:15] 


The question of what proportion of federal granting agency funding should be 
directed towards funding basic, applied and strategic research is a contentious issue that 
is often raised by scientists, funding agencies and governments around the world: 


We ask this question for 50 years now, since government started investing in 
science and technology: What is the balance between fundamental research and 
applied research? There's no magic formula to answer this ... if you [look at] the 
funding council[s], a third of their funding is actually devoted to a kind of strategic 
research ... ls that too much? Scientists say so. Government thinks no … | think we 
should not go too far in that direction. [Benoit Godin, Director, Observatoire des 


sciences et des technologies, 66:10:25 | 


The Committee recognizes that there is no consensus on this issue. 


Federal Research Priorities 


th the issue of what proportion of federal funds should be directed towards 
arch in target areas deemed to be of national importance is the question 
e deciding on which areas are to be funded. Target areas in strategic 
deral granting agencies are generally proposed by multistakeholder 
cided upon by the agencies’ governing councils, or in the case of the 
me critics point to the lack of clear research priorities at the federal 
curring debate that relates to the subject of the allocation of 
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| would suggest that one of the causes of this problem is simply that we do not have 
a clear set of priorities for our country. As a consequence, we try to fund all fields of 
research and thus cut a small pie into even smaller pieces. To be fair, the 
government has tried to remedy this situation by setting up special funds, programs 
and even institutions to focus scarce research funds in specific areas, but at the end 
of the day we still do not have these priorities. Rather, we have all-encompassing 
generalities such as economic development or social capital. [J. Adam Holbrook, 
Associate Director, Centre for Policy Research on Science and Technology, Simon 
Fraser University, 66:09:10] 


Some national governments do channel relatively large (according to some 
members of the research community) proportions of funds for university-based research 
to target areas of national importance. For example, in January 2002, the Australian 
government directed the Australian Research Council (Australia’s main funding agency 
for basic research) to allocate 33% (about $130 million) of its total funds for the 
2003 funding round to research projects in four priority research areas: nano-materials 
and biomaterials, genomics and gene expression, complex/intelligent systems, and 
photonics. The directive met with some criticism from the research community, but others 
feel that for small countries, such national priority setting is necessary: 


It’s certainly an issue which is very contentious, but the argument in favour of it in 
relation to Australia is that we have a very small population. We have a large 
landmass, but in terms of the population, the spread of our expertise, we can't 
possibly afford to cover everything and priority setting at some stage is inevitable. 
[Fiona Wood, University of New England, Australia, 79:19:55] 


In terms of decisions on the allocation of federal research funds in Canada, the 
Committee shares the concerns expressed by some of the witnesses about the lack of a 
clear set of federal priorities in this area. The Committee believes that a stronger federal 
S&T advisory framework is necessary to ensure that the government is receiving 
appropriate and adequate counsel for setting federal research priorities and policy. Such 
advice would ensure that the government is providing sufficient funding for research in 
strategically important areas while at the same time maintaining a strong support base for 
other research. The Committee was encouraged to see that the issue of decision making 
and priority setting is mentioned in the federal government’s Innovation Strategy, and that 
the government will consider establishing a national, arm’s length science organization 
(the Canadian Academies of Science) to provide independent assessments on 
science-based issues of national importance. The Committee notes that other 
industrialized countries have S&T advisory frameworks that include a Chief Scientific 
Adviser to the executive branch of government who is usually head of a government 
office of science and technology policy (e.g., in the United Kingdom and the United 
States). Some countries also have priority-setting programs that identify future S&T 
directions and needs (e.g., the United Kingdom’s Foresight program). In terms of 
Canada’s S&T advisory framework, the Committee recommends: 


See http://www. innovationstrateaqy.gc.ca/cmb/innovation.nsf/vVRTF/PDE/S$file/achievina. 


D 70: 
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RECOMMENDATION 6 


That the Government of Canada establish a more formalized 
mechanism, in consultation with the provinces and territories, for 
setting or modifying S&T policy, deciding on funding priorities, and 
ensuring that they are implemented. Such a framework could include 
a science advisory body and/or Chief Scientific Adviser that would 
report directly to Parliament. 
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CHAPTER FOUR: OTHER PLAYERS IN CANADA’S 


INNOVATION FRAMEWORK 


During the hearings, concerns were expressed that important contributors to 
Canada’s “innovation framework” are not receiving adequate consideration under the 
present system for allocating federal research funds. Two categories of stakeholders in 
particular were discussed: highly qualified personnel and college researchers. In this 
chapter, the Committee addresses how the federal government, via the granting 


agencies, can help to realize the potential that these two groups have in building 
Canada’s R&D capacity. 


Highly Qualified Personnel 


Many witnesses described the importance of investing in the training of highly 
qualified personnel to ensure that Canada has the supply of skilled researchers 
necessary to remain competitive and move ahead on global innovation scales: 


... [MJaybe the more important outcome of NSERC funding is the highly trained 
people we produce. These are the people who go on to work in universities, in 
industry, and in government. And they’re the people who will have ideas of the 
future. They will be the people who create new companies and other real 
innovations. | think its been identified that in order for Canada to move ahead of 
other countries in this area, it’s the people and their skills that we really need. 
[Elizabeth Boston, NSERC, 39:17:10] 


Some witnesses expressed concern that Canada will not have enough skilled 
researchers to advance or even maintain the knowledge-based economy in future 
years, especially in emerging areas of research. These witnesses pointed to the 
problems that many universities have in attracting top-quality students into graduate 


studies: 


I’m more concerned about the fact that we are not attracting the best people into our 
PhD programs. PhD stipends, or salaries if you like, are so uncompetitive in my field 
[design engineering] compared to industrial salaries that it’s absolutely economically 
a disaster to do graduate studies. [Peter Frise, Professor, University of Windsor, 


58:10:50] 


The training of highly qualified personnel to provide the next generation of skilled 
researchers is part of the missions of all three federal granting agencies. Competitive 
scholarship and fellowship programs exist at all three agencies; they generally consist of 
a review of proposals by the applicants’ universities, and, for most programs, review by a 
selection committee of researchers (appointed by the granting agencies) of all 
applications received from the universities by the agencies. For fiscal year 2000-01, 
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expenditures on the training of highly qualified personnel totalled $96.9 million at NSERC 
(about 19% of its budget), $33.5 million at SSHRC (approximately 26% of its budget) and 
$33 million at CIHR (about 9% of its budget). 


In March 2002, NSERC announced that part ($6.5 million) of the 7% (or 
$36.5 million) increase to its annual budget beginning in fiscal year 2002-03, which was 
announced in the federal budget of 2001, would be channelled into its training programs 
for highly qualified personnel. The CIHR’s training budget will increase in 2002 with th= 
introduction. of the Strategic Training Initiative in Health Research. The initiative, which 
began on a small scale in fiscal year 2001-02 with contributions from CIHR of 
approximately $1.7 million, will be in full operation in fiscal year 2002-03 with a CIHR 
allocation of $14.3 million. The initiative will provide training grants (up to $300,000 per 
year) to institutions to pay for stipends and travel for trainees (undergraduate through 
postdoctoral level) conducting research in all areas of health research. Preference will be 
given to innovative, interdisciplinary programs, and applications are especially sought in 
areas where the applicants can demonstrate the need to develop research capacity. 


At NSERC, scholarships for undergraduate and graduate (Master's and PhD level) 
students are available, as well as fellowships for postdoctoral fellows. At all levels, 
scholarships and fellowships are available at NSERC that allow students to conduct some 
or all of their research in an industrial setting. At SSHRC, a doctoral and postdoctoral 
fellowships program is offered. Doctoral research awards, combined MD/PhD 
studentships, and a variety of fellowship programs for postdoctoral researchers and 
health professionals are offered by CIHR. 


Success rates and stipend levels vary among programs and agencies. At the 
graduate level, the annual stipend ranges from approximately $17,000 to $19,000 and the 
success rate (data from competition year 2001) ranges from a low of about 38% at 
SSHRC to a high of approximately 65% at the NSERC. The success rate for obtaining an 
NSERC Industrial Postgraduate Scholarship, where scholars spend a minimum of 20% of 
their time at a sponsoring company on activities related to their thesis project, is 

considerably higher (in the range of 90 to 95%).*’ Stipends for postdoctoral fellowships 
range from $35,000 per annum for two years at SSHRC and NSERC, to $38,500 (for 
PhD holders) or $48,500 (for health professionals) at CIHR. The success rate for 
obtaining postdoctoral fellowships ranges from a low of 25% at SSHRC to a high of 
37% at NSERC (data for competition year 2001). The success rate for receiving an 
NSERC Industrial Research Fellowship, which provides financial contributions to support 
recent doctoral graduates engaged in industrial research, is much higher — in the range 
of 80%.°° Since an initial “triage” of applications occurs at the universities for most of 
these awards, the actual success rates are generally lower than the figures reported here. 
Students and postdoctoral fellows are also supported through the research grants of 


% NSERC contributes $13,800 a year (for two years) for each Industrial Postgraduate Scholarship and the 


sponsoring company contributes a minimum of $5,500 a year. 


°° NSERC contributes $30,000 a year (for two years) for each Industrial Research Fellowship and the sponsoring 


company contributes a minimum of $10,000 a year. 
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individual researchers, but the annual stipends are usually smaller than those listed 
above. Representatives of SSHRC point to the relatively low success rate of its doctoral 
fellowship Program and the lack of a Master’s level scholarship program as further 
evidence that it is under-funded in comparison to the other granting agencies. 


The Committee appreciates the importance to Canada’s future in R&D and to its 
economy of attracting more students to graduate level study. In addition, the Committee 
believes that more efforts should be made by the federal government at earlier stages in 
the training “pipeline” to attract students to scientific research. It notes that NSERC 
already has a program for undergraduate students, the Undergraduate Student Research 
Awards Program, which provides stipends for students to conduct research for 
four-month periods in universities or in industry. The Committee is also in favour of the 
federal government increasing its role in the promotion of science and the impact of R&D 
to youth and to the public at large. It is aware of programs that are already in place at the 
agencies to promote science or to communicate the results of federally sponsored 
research to the media and general public. For example, the PromoScience Program at 
NSERC supports programs that promote science to school-aged (elementary, high school 
and cégep level youth). It funds practical, interactive training programs (e.g., science 
camps, university science outreach programs, science clubs, and programs that involve 
students in research). The Committee encourages the government to explore additional 
avenues (either via the granting agencies or other bodies) for expanding its role in this 
type of science promotion. 


The federal government's Innovation Strategy also recognizes that developing, 
attracting and maintaining highly qualified personnel is crucial for Canada’s innovation 
performance.“ Among other initiatives, the strategy calls for increased financial 
incentives to attract students to graduate level study. As part of the strategy, the federal 
government announced a $125 million endowment in February 2002 for the Advanced 
Research in the Humanities and the Human Sciences Fund to fund up to 100 doctoral 
fellowships and mid-career awards to exceptional researchers in certain areas of the 
humanities (e.g., human rights and social justice) that reflect the interests of former Prime 
Minister Pierre Trudeau. Some members of the Committee appreciate the additional 
investment in training that the government has made via the Pierre Elliott Trudeau 
Foundation, although the Commitee notes that the monies will support relatively few 


individuals in a limited number of research areas only. 


ers of the Committee expressed reservations about the 
on ee ate federal government's injection of $125 million into the Pierre Elliott 
Trudeau Foundation for doctoral and postdoctoral fellowships in the humanities and 
human sciences. The reservations expressed by these members arise primarily from the 
fact that the federal government funds released for this initiative should have gone directly 
to SSHRC, a public body, and not to a private foundation, which is not held accountable 
to the public for the funds it receives. Given the chronic under-funding of SSHRC, such 
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an injection of public funds into its coffers would have been particularly welcome. The 
Committee believes that applications for these fellowships should be peer-reviewed 
based on the excellence of the applicant’s record and quality of the proposal. 


One witness made the following observation with respect to the training of highly 
qualified personnel and their importance to the success of any innovation strategy: 


Research is the practice of turning wealth into ideas. Innovation is the practice of 
turning ideas into wealth. And education is the practice of turning people into citizens 
who can be innovative and generate more wealth. In my view, all three of these 
issues: research, innovation and education, are inextricably linked and to treat one 
without the other is not going to generate success for Canada’s future. [Peter Frise, 
Professor, University of Windsor, 58:09:30] 


The Committee shares these sentiments and believes that Canada will not make 
progress in its goal to become one of the most innovative countries in the world without 
increased investment in the training of highly qualified personnel. The current level of 
investment allows the agencies to offer scholarship and fellowship programs that have, 
in most cases, relatively low success rates and uncompetitive stipends. The Committee 
suggests that additional investments in research training should be directed through 
established training programs at the granting agencies that are accessible to a broad 
cross-section of the student and postdoctoral fellow population. As such the Committee 
recommends: 


RECOMMENDATION 7 


That the Government of Canada, given the increases in overall funding 
levels that should be made to the agencies, encourage the three 
granting agencies to increase success rates and stipend levels for 
existing graduate scholarship and postdoctoral fellowship programs. 
The granting agencies should also create or enhance scholarship 
programs that introduce undergraduate students to research. 


The Committee suggests that the granting agencies direct a portion of any additional 
funds earmarked for training to the enhancement or promotion of scholarship and 
fellowship programs in which students or postdoctoral fellows can collaborate on 
research projects with other stakeholders not based in universities (e.g., industry 
researchers). The Committee believes that such partnerships are essential to improving 
and diversifying the skills of highly qualified personnel. 
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Colleges 


In terms of contributions to R&D and innovation, Canada’s community colleges“° 
play an important role in applied research, technology transfer and product development. 
Much of the technology transfer work occurs at the community level with local businesses 
and colleges working together. Colleges also play an important role in education and 
research in areas of the country where there are no universities. For example, in 
Canada’s North, the three territorial northern research institutes*' are a major focus for 
northern _fesearch activities. A representative from the Association of Canadian 
Community Colleges (ACCC) testified that the potential of colleges to contribute to 
Canada’s R&D base goes largely unrecognized by the government: 


... [The government] continues to marginalize the important contribution of colleges 
to innovation, research, and the technology transfer needs of business, industry, 
and community organizations ... We continue to wonder why the recognition and 
funding are disproportionate to the breadth, depth, and economic impact of the 
applied research emanating from the college system. [Gerald Brown, President and 
Chief Executive Officer, Association of Canadian Community Colleges, 43:09:15] 


Additionally, colleges feel excluded from many of the programs offered by the 
federal granting agencies. Historically, college researchers have not been a major focus 
of granting agency programs. For most programs at SSHRC, applications are accepted 
from any researcher affiliated with a post-secondary institution. For its two institutional 
grant programs, an institution must be a member of the Association of Universities and 
Colleges of Canada (AUCC) to be eligible to apply for support (AUCC members are 
universities and university colleges whose primary mission is to provide university degree 
programs). Until recently, college researchers were not eligible to apply for any of 
NSERC’s programs. In 1999, NSERC expanded its eligibility criteria so that college 
researchers from eligible institutions can now apply, as co-applicants with university 
researchers, for certain types of project research funding. As of June 2002, four 
community colleges had been declared eligible to participate in NSERC programs: the 
British Columbia Institute of Technology, the New Brunswick Community College at 
Bathurst, the Nova Scotia Community College — Annapolis Campus, and Old’s College 
in Calgary. NSERC has yet to receive applications for funding involving the participation 
of researchers from other colleges. At CIHR, only researchers at Canadian universities or 
affiliated institutions can apply as principal applicants for CIHR grants. The agency is 
considering a significant broadening of eligibility to include any health researcher working 
in the not-for-profit sector (with the exception of the federal government). 


Even when college researchers are eligible to apply for the granting agencies’ 
programs, ACCC believes that researchers from colleges cannot compete with their 


Joe) eee 
40 The term “community college” inclu 

i College (NWT), and 
41 The Northern Research Institute of Yukon College, the Aurora Research Institute of Aurora College ( ), an 


the Nunavut Research Institute of Nunavut Arctic College. 


des institutes, and cégep (in Québec). 
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university counterparts because of the programs’ selection criteria that reflect the 
university, not college, environment: 


Within the peer-review process, the colleges and institutes are overshadowed by the 
university-focused process. This process is based solely on university criteria ... At 
the practical level, involving college faculty in applied research has been and 
continues to be a huge challenge, due to the ... high teaching loads of college and 
institute faculty. [Gerald Brown, President and Chief Executive Officer, Association 
of Canadian Community Colleges, 43:09:15] 


A representative from AUCC suggested that to make the granting agency programs 
more accessible to college researchers and to reflect the kind of research conducted at 
community colleges, the structure of programs and selection criteria would have to be 
altered: 


[I]t may be that the kind of research [that the colleges and institutes] are doing would 
not fit within the criteria or the guidelines of the granting councils, and if there’s a 
decision to proceed to support colleges, these would have to be adapted to meet the 
needs of the colleges. [Robert Giroux, President and Chief Executive Officer, 
Association of Universities and Colleges of Canada, 51:10:35] 


As an alternative to altering current eligibility and selection criteria at the granting 
agencies, ACCC proposes that a separate funding pool, what it calls the Canadian 
College Innovation and Technical Assistance Program, be established to help support the 
applied research, technology transfer and commercialization work that is carried out by 
Canada’s community colleges. The components of the proposed program include Chairs 
for college researchers, fellowships and internships for students, networks of centres of 
excellence with college and industry participants, and a fund to promote and assist the 
development of commercial products and processes. The ACCC suggests that the 
program have an initial duration of five years and that it would require about $600 million 
of funding over that period. 


The Committee believes that altering program eligibility criteria at the granting 
agencies to allow more access to college researchers would be a symbolic change only; 
college and university researchers work in quite different environments, and college 
researchers likely would have difficulty competing with their university counterparts in 
competitions based on present selection criteria. Similarly, the Committee is of the 
opinion that it would be difficult for the granting agencies to adapt the selection criteria for 
most granting programs so that the criteria reflect the situation for both university and 
college researchers. Instead, the Committee supports ACCC’s idea of establishing 
discrete funding programs for college researchers and students, and thus recommends: 
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RECOMMENDATION 8 


That the Government of Canada create separate research funding 
programs administered either by the granting agencies and/or other 
government bodies to support college researchers and students. The 
programs’ selection criteria should reflect the situation at colleges 
and be based on excellence. Appropriations should be made to the 
relevant agencies to deliver these programs. 


The Committee is of the opinion that it would be premature to suggest a funding level 
and structure for such programs at this time; the federal government and ACCC should 
work together to plan and develop any initiative. Although the Committee believes that 
separate funding programs at some level are necessary for college researchers, it also 
feels that the granting agencies should continue to work to include college researchers 
in existing programs that support applied research. The Committee strongly believes 
that the inclusion of college researchers in these programs will increase collaboration 
between university and college researchers, and ultimately boost Canada’s innovation 
performance. 
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CHAPTER FIVE: IMPROVEMENTS TO THE 
SYSTEM FOR ALLOCATING FEDERAL 


RESEARCH FUNDS 


The Committee supports the practice of employing peer review as a mechanism 
for determining the allocation of federal research funds, but it believes that a number of 
improvements can be made to the system. This chapter addresses this issue and builds 


on testimony heard during the hearings on “best practices” for peer review, and for the 
allocation of federal research funds in general. 


Perceived Weaknesses in the System 


In addition to the broad areas of concern expressed earlier in the report, the 
Committee is worried about other shortcomings in the present system for the allocation of 
federal research funds. These areas of concern include unsatisfactory feedback to, or 
appeal mechanisms for, applicants; a lack of data on the efficacy of peer review in 
general; insufficient internal reviews of agency programs; peer reviewer overload; and 
inadequate efforts by the granting agencies to measure and communicate the impacts of 
federally funded research on Canadian society. 


Inadequate or Inconsistent Feedback to Applicants 


The Committee believes that the granting agencies could place more effort into 
improving and harmonizing the types of feedback that applicants receive following a 
funding decision, and ensuring that a formal, transparent appeal mechanism is available 
at each agency to deal with complaints about the decision-making process. 


The Committee notes that all three agencies provide some type of feedback to 
applicants following a funding recommendation, but that the feedback provided varies 
within and among programs, and according to the agency involved. The Committee heard 
that the feedback obtained from the granting agencies is not always very useful: 


[The feedback] is inconsistent. In some cases, yes, there is [adequate feedback], 
and people are able to improve their applications and are successful in a 
subsequent submission. In some instances, it isnt as helpful as it might be. In part, 
I’m sure this reflects the staff pressures the granting agencies are under in trying to 
get comments out. But it can be difficult when you have one or two external reviews 
that are very positive, yet the decision is negative. One is left in a de vas 
how one is going to improve [the application]. [Wayne Marsh, President, Canadian 
Association of University Research Administrators, 43:10:15] 
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By law, applicants have access to their application files (however, under the 
Privacy Act, the written opinion of a reviewer about an application is available to the 
applicant, but the name of the reviewer cannot be divulged). For its major Discovery 
Grants Program, NSERC sends a notification of funding decision to applicants, followed 
shortly thereafter by the selection committee’s comments on the application, if available. 
Applicants receiving comments also receive any external referees’ reports on the 
application. When selection committee comments are not available, an applicant may 
submit a written request to NSERC to obtain any external referees’ reports received on 
the application. For SSHRC’s Standard Research Grants Program, applicants receive 
copies of all information used to make a decision, including external written assessments 
and the committee’s deliberations relating to their application. For its major research 
funding programs, CIHR provides copies of external referees’ reports (if available), copies 
of the internal reviewers’ assessments (one from each of the two principal readers 
assigned to the application), and a summary of the committee discussion (when 
available) with the notification of a funding decision. 


At NSERC and SSHRC, a formal appeal process is in place for applicants who feel 
that their applications have been unfairly treated by the peer-review process. At NSERC, 
appeals of decision must be based on compelling evidence of error or discrimination in 
the review process; the onus is on the applicant to demonstrate the error. The process 
may vary according to the program, but for the Discovery Grants Program external 
consultants, who are senior members of the research community and have some 
experience with NSERC peer review, examine appeals: 


Sometimes errors can occur in the peer review process ... and if an applicant feels 
their application wasn’t assessed appropriately, they can use our appeal process to 
request a review of the decision. NSERC then requests an independent review by a 
senior researcher, who was not involved in the original decision, and staff makes a 
final decision based on this adviser’s report. [Elizabeth Boston, NSERC, 39:15:40] 


At SSHRC, appeals of decisions must be based on procedural or factual error in the 
process. If SSHRC determines that there are sufficient grounds for appeal, the 
application and any new information provided are examined by the adjudication 
committee that made the original decision. The two agencies receive a relatively small 
number of appeals each year (see Table 2). 
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Table 2 
Data on Applications Appealed at NSERC and SSHRC 
for the Agencies’ Major Research Grants Programs 
Competition Years 2000 and 2001 


OS) SAIC NE CRT M. 


C bp N % of No. of 
Agency ompetition One No. of Applications Successful 
Year Appeals Applications Appealed Appeals 
NSERC” 2000 132 2963 4.45 18 
2001 96 3089 oti 15 
SSHRC*? 2000 10 1571 0.64 1 
2001 11 1821 0.60 1 


The CIHR does not have a formal appeal process. However, for CIHR’s major 
research funding programs, applicants who are unsuccessful in one competition may 
resubmit the same (or a similar) application in a subsequent competition, and may include 
a two-page rebuttal with the resubmission to address concerns raised by reviewers or to 
counter criticisms that the applicants believe are unfounded. The Operating Grants 
Program at CIHR has two competitions per year (unlike the major grants programs at 
NSERC and SSHRC that are held once a year), thus the time between receiving a 
notification of decision and resubmission is a matter of a few months. The Committee 
appreciates the process at CIHR whereby applicants can submit rebuttals to reviewers’ 
comments with a subsequent application, but it is concerned that CIHR does not have a 


formal appeal process for its programs. 


The Committee notes that some foreign granting agencies (e.g., the Australian 
Research Council) allow applicants to reply to external referees’ comments before a 
funding recommendation is made. It also notes that the Cross Council Research Forum 
of the U.K. research councils suggests that allowing applicants the “right to reply” to 
referees’ comments should be part of joint, cross-council programs that review 
interdisciplinary proposals. The Committee encourages the Canadian granting agencies 
to consider the feasibility of incorporating such a “rejoinder mechanism” in its major 


granting programs. 


The Committee is aware that providing feedback and offering a formal appeal 
effort on the part of the agencies and selection 
financial cost involved in providing such feedback. 
hat the agencies should make every effort to provide 


process necessitates substantial 
committees, and that there is a 
Nevertheless, the Committee feels t 


oe 


42 Data for NSERC’s Research Grants Program (now called the Discovery Grants Program). 


43 Data for SSHRC’s Standard Research Grants Program. 
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applicants with as much feedback as possible following a funding recommendation. Since 
all three agencies are conducting more of their business electronically, the administrative 
burden and cost of providing feedback may decrease in the future as “e-business” 
practices become more established and new technologies become available (the 
Committee recognizes that privacy issues may limit some of the efficiencies that could 
potentially be realized by providing feedback electronically). The Committee 
recommends: 


RECOMMENDATION 9 


That the Government of Canada ensure that the granting agencies 
release all information on file relevant to a funding recommendation to 
applicants in addition to the notification of decision. Additionally, a 
formal appeal process, limited to perceived errors in procedure or 
fact, should be in place for applicants to all peer-reviewed programs, 
and a third-party, not the original selection committee, should review 
appeals of decision. 


Peer Review Is Untested 


2] 


Peer review is often described as being “rigorous” or “a cornerstone of excellence.’ 
According to one witness, however, the value of peer review for deciding on the allocation 
of research funds is taken as an “article of faith,” and the system is largely untested: 


... [WJe see that there is little scientific evidence for the efficacy of peer review in 
general ... Good science means, at the very least, conducting reliable, repeatable 
scientific research on both consistency of past reviews and the impacts of decisions 
taken. [Bryan Poulin, Professor, Lakehead University, 58:09:30] 


The witness argued that agency databases should be opened to interested researchers 
so that these individuals can study whether the system is working, and went on to 
suggest that: 


... [Funds] should be more widely distributed until we find out if the peer review 
system is fatally flawed. If it’s fatally flawed, it needs an overhaul. [Bryan Poulin, 
Professor, Lakehead University, 58:09:30] 
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à gx WE Committee shares the concerns about the paucity of data on the efficacy and 
Fe of peer review in the Canadian system. It notes, however, that comprehensive 
studies or reviews of peer review practices have been conducted in such countries as the 


iC : 
United States” and Australia.“ One witness suggested that there is a large literature on 
peer review: 


In terms of the literature on peer review, it’s vast and disaggregated. There's a long 
history since the funding councils have been around of [studies] regarding the 
Strengths and weaknesses of peer review. Some being based on anecdotal 
comments, some have been based on studies done at different times and under 
different circumstances. There are few systematic studies linking grant decision 
outcome with funding policies. The majority of studies that are actually represented 
in the literature are not by those who've been involved in the funding councils 
-themselves. They tend to be produced by those who are, like myself, independent 
researchers. [Fiona Wood, University of New England, Australia, 79:19:55] 


Internal Evaluation of Agency Programs and Practices May Be Insufficient 


The Committee believes that the agencies themselves should be doing more to 
evaluate their programs and practices, including peer review, to ensure that they are 
efficient, transparent and responsive to the needs of the research community. The 
Committee notes that internal program evaluation studies are conducted by the agencies, 
but that the period of time between evaluations is often long (in some cases more than 
10 years), even for major funding programs. At present, both the Discovery Grants 
Program at NSERC and the Operating Grants Program at CIHR*’ (the two agencies’ 
major research grants programs) are under evaluation. 


The Committee points to the evaluation plan proposed by the CIHR as being a 
good model for internal program evaluation. The agency plans to evaluate all programs 
on a periodic basis: the aim is to evaluate continuing programs every five years, 
limited-term strategic programs at the end of their term (normally five years), and 
partnered programs, which also normally have a term of five years, at the termination of 
their memorandums of understanding. In addition, the entire organization will be subject 
to international review every five years, and the performance of individual institutes will be 
evaluated at the time of the appointment (or renewal) of their scientific directors (every 


four years). 


Swear CORRE TRE PSN TE HE EX HET 
#  D.E. Chubin and E. J. Hackett, Peerless Science: Peer Review and U.S. Science Policy, State University of 


New York Press, Albany, New York, 1990. 
Review Process, report commissioned for the National Board of Education, Employment 
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ASA aa Sek Australian Government Publishing Service, Canberra, 1997. 


and Training (Australia ; 
endix 3 for a list of recent major evaluation studies at the three agencies. 
g Grants Program since it has never been 


4 See App 


47 The Auditor General recommended that CIHR evaluate its Operatin 
subject to an intensive evaluation. 
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The Committee suggests that internal examinations of peer review by the agencies 
can be conducted in isolation and do not have to form part of more intensive program 
evaluations. For example, the Committee notes that the Director of the National Science 
Foundation (NSF) in the United States must submit an annual report on the NSF proposal 
review system. The report provides summary information about levels of proposal and 
award activity and the process by which proposals. are reviewed and awarded. The report 
is posted on the agency's Web site each year.” 8 In other countries, funding agencies 
(e.g., the Economic and Social Research Council in the United Kingdom) conduct 
periodic, independent reviews of their peer review processes. The Committee notes that 
evaluations of peer review practices have occasionally been conducted by the Canadian 
federal granting agencies (see Appendix 3). As such, the Committee recommends: 


RECOMMENDATION 10 


That the Government of Canada require the granting agencies to 
engage in more regular internal reviews of their own programs and 
practices (including peer review), and to periodically examine 
decision-making processes at other Canadian and foreign agencies to 
ensure that best practices for the allocation of research funds are in 
place. The results of these internal evaluations should be easily 
accessible to the research community and general public. 


Peer Reviewer Overload 


As mentioned earlier in the report (Chapter Three), it is often difficult to find 
qualified, arm’s length external referees or selection committee members to evaluate 
proposals in emerging and interdisciplinary areas because of the small number of 
researchers working in those areas. Members of these small communities tend to know 
and work with each other; the Canadian federal agencies all have guidelines intended to 
prevent a researcher that has a conflict of interest with an applicant from acting as a 
referee of that applicant's proposals. Because of the relatively small research community 
in Canada, finding impartial reviewers for more “mainstream” research proposals may be 
problematic as well. Additionally, the introduction of new, peer-reviewed programs at the 
federal agencies, government departments and other organizations (e.g., the Canada 
Foundation for Innovation) has led to an increased demand for peer reviewers, and to a 
phenomenon termed “peer reviewer fatigue” by some commentators. 


The federal granting agencies have tried to reduce the problems associated with a 
small reviewer pool by calling on international reviewers to participate in the review 
process: 


Report to the National Science Board on the National Science Foundation’s Merit Review System Fiscal Year 
2000, http://www.nsf.gov/nsb/documents/2001/nsb0136/nsb0136.pdf 
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The potential for peer fatigue is quite critical. It’s particularly true in a country like 
Canada, compared to the States, where there are a limited number of qualified 
experts who can serve on review panels. Countries such as Sweden, Australia, and 
New Zealand make substantial use of international experts to ensure the 
independence of their review processes and to counteract peer fatigue. Certainly, 


I'm aware of international experts participating in Canadian reviews, and | think 
that’s an increasing trend. [Alan Winter, President and CEO, New Media Innovation 
Centre and Council of Science and Technology Advisors, 55:09:35] 


The agencies also limit the number of times any one individual can be called on by 
an agency program to act as an external referee. However, since granting agencies 
around the world are experiencing the same problems, other solutions to reduce overload 
on reviewers are probably required. One option being employed by some agencies is 
offering some form of incentive to reward reviewers or their universities: 


Peer reviewers in the past have tended not to be paid, so it’s interesting to note that 
a number of funding councils are moving much more towards the idea of paying for 
the reviews that they receive. The EPSRC [Engineering and Physical Sciences 
Research Council] in the U.K. is a good example of this. The concern is basically 
driven by the perception of overload on reviewers so the idea is in fact, given their 
competing demands on the time of the best reviewers that an incentive needs to be 
provided to ensure that there is a value placed on the reviews that are received. 
[Fiona Wood, University of New England, Australia, 79:19:55] 


In EPSRC’s “Referees’ Incentive Scheme,” which began as a three-year pilot project in 
2001, university departments earn points for “useful” referee reports returned on time to 
EPSRC by researchers. In December of each year, beginning in 2002, points 
accumulated by departments over the preceding academic year will be translated into a 
share of the scheme fund, which stands at £750,000 (approximately $1.7 million) for the 
first year. These funds will be paid centrally to institutions on behalf of departments, and 
heads of department can use the monies for any purpose that EPSRC would normally 
consider to be legitimate expenditure on a grant. Other granting agencies also use 
some form of payment to recognize and reward the work of reviewers. In Canada, for 
example, the Alberta Heritage Foundation for Medical Research offers payment to 


external referees for reviews: 


| should point out that because we work outside the province, in other words we're 
pulling in our reviewers from around the world, there's really no reason for them to 
help Alberta, other than altruism, and altruism only goes so far. So we actually pay 
for this, which adds to the cost of our peer review system, but | think it increases the 
quality, because we are pulling in an international opinion. [Matthew Spence, 
President and Chief Executive Officer, Alberta Heritage Foundation for Medical 


Research, 66:10:50] 


ally offer no compensation to peer reviewers. 


ting agencies gener 
The three federal granting 29 ent only for expenses incurred to attend 


Members of selection committees receive paym 
committee meetings. 
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An informal survey on the topic of "peer reviewer fatigue” by NSERC in 
2000 suggests that, at present, it may not be a major problem.” The Committee 
appreciates the difficulties associated with finding individuals to act as referees for grant 
applications. It notes, however, that the federal agencies are augmenting the amount of 
business that they conduct electronically, and that one of the stated goals of “e-business” 
is to reduce the workload for peer reviewers. Given the considerable expense involved in 
offering some form of payment for referee reports, and that peer reviewer workload does 
not seem to be unmanageable at present, the Committee is reluctant at this time to make 
a recommendation that focuses on payment of referees. Instead, the Committee 
encourages the agencies to continue with their efforts to find other ways to reduce the 
workload associated with peer review that would not involve large increases to their 
administrative budgets. Other options, including payment of referees, may have to be 
considered if these efforts do not have the desired result or if the workload linked to peer 
review continues to increase. 


Outcomes and Impact of Research Are Not Adequately Measured and 
Reported 


The Committee is concerned that the Canadian federal granting agencies do not 
put enough effort into measuring and communicating the outputs, outcomes and 
impacts” of federally funded research programs. Funding agencies around the world are 
being asked to better measure and report on the outputs, outcomes and impacts of 
funding. The under-reporting of output measures by granting agencies may be related to 
several “disincentives” for reporting on performance: 


For high-risk research but even for less risky research, there will be many instances 
where the original research objectives were not met. Some oversight organizations 
could view this as failure. In addition, bibliometric studies have shown that ... 
seminal research is produced by relatively few performers. That is independent of 
whether the metric is the number of papers you produce, the number of patents, the 
number of citations, or whatever, especially for outputs. They are the quantification 
of the near-term products. Why would organizations be motivated to show the 
concentration of productivity in a relatively small number of performers? [Ronald N. 
Kostoff, 88:11:10] 


A form of “performance monitoring” is undertaken by the Canadian granting 
agencies through the annual Departmental Performance Reports tabled in Parliament. 
Since 2001, these reports are supposed to place more emphasis on linking resources to 
outcomes (i.e., benefits to Canadians), rather than reporting largely on departmental 
activities. Performance monitoring is also undertaken internally by the agencies through 
irregular evaluations of individual programs. In addition, independent performance audits 


“8 NSERC Contact, Fall 2000, Vol. 25, No. 3, http://www.nserc.ca/pubs/contact/v25_n3_e.pdf 


°° Output = the direct result of program activities; outcome = accomplishment of program objectives attributable to 


program outputs; and impact = broad (often long-range) social, economic or environmental results of a research 
program. Definitions adapted from categories discussed in the U. S. Government Performance and Results Act 
of 1993. 
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(as well as financial and compliance audits) are conducted for Parliament by the Office of 


the Auditor General, which periodically assesses the “value for money” of some of the 
agencies’ programs. 


At the level of the individual applicant, peer review evaluates the outputs and 
outcomes (but rarely the impact) of past research funding provided by the agency. At the 
level of the agency, other measures and types of performance evaluation are necessary. 
There are different sorts of measures, whose utility varies according to the type of 
research and discipline involved, that can be used as indicators of agency performance. 
Bibliometrics, the study of the quantitative data of the publication patterns of individual 
articles, journals, and books in order to analyze trends and make comparisons within a 
body of literature,”’ is used by some researchers and a few agencies to measure the 
outputs, outcomes and impacts of research programs. In bibliometrics, counts of numbers 
of publications are taken as a measure of research output, and citation data (the number 
of times a paper is cited in the literature) are used to measure impact. Bibliometrics can 
be used to monitor or evaluate a group's (e.g., an institution, agency or country) research 
productivity and impact. The Director of a Canadian organization that conducts 
bibliometric studies as part of its activities, informed the Committee that, in terms of 
program evaluation: 


CIHR has asked us in the last year to evaluate their granting 
programs ... everything [from] the scientific production [to] the quality of the papers 
published by the researchers who have received grants. This is the first time that 
CIHR asked us, and the other Councils do so, | would say, quite sporadically. 
[Benoit Godin, Observatoire des sciences et des technologies, 66:09:25] 


The Committee recognizes that for program evaluation purposes, the data 
produced from bibliometric studies should not be taken at face value nor used in isolation, 
since there are problems associated with such data. For research with commercial 
potential or purpose, other measures (e.g., the number of patents and licences issued, 
the production of spin-off companies etc.) related to the economic impact of the research 
can be examined instead of, or in addition to, traditional bibliometric indicators to assess 
the impact of research. Some critics argue that measuring the impact of research in the 
humanities and social sciences is difficult, if not impossible, to do, and that performance 
indicators are suited more to measuring the impact of research in the applied sciences 
and technology. The design and utility of performance indicators for the social sciences is 
currently a major subject of debate in that community. Despite the difficulties in measuring 
the impact of research in the social sciences, one witness suggested that efforts in 
designing and using performance indicators should be stepped up: 


was an interest in impact measurement. But it seems as 


o, there 
Ee ses is that it is too difficult to measure. The task requires 


though what is being said 


51 {SI (formerly the Institute for Scientific Information) definition, 


http:/Amww.isinet.com/isi/search/glossary/index.htm| 


52 sR. Barré, “Sense and nonsense of S&T productivity indicators,” Science and Public Policy, Vol. 28, August 


2001, p. 259-66. 
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instruments that do not exist and there certainly are major methodological 
challenges. It is certainly not easy to measure the social or cultural impact of 
scientific activities, but it is our belief that a community effort, probably supported by 
government programs, should be made with regard to this important issue of the 
measurement of non-scientific impacts of ... science and technology. [Benoit Godin, 
Observatoire des sciences et des technologies, 66:09:50] 


The Committee notes that NSERC did provide some quantification (via 
bibliometrics, and numbers of patents, licences and spin-off companies produced from 
NSERC sponsored research) of the outcomes and impact of agency funded research in 
its 2000-01 Departmental Performance Report? many of the figures provided refer to the 
nation’s performance, thus the exact contributions of NSERC funded research to these 
figures and the impact of particular programs is difficult to ascertain. The Departmental 
Performance Reports produced by SSHRC and CIHR for the same year provide fewer 
quantitative data, and, in the case of SSHRC, less concrete qualitative information on the 
link between funding provided and the outcomes and impacts of that funding. The 
Committee notes that SSHRC plans to introduce a “Final Research Report” form that 
grantees will be required to complete at the end of each granting period. The form will 
capture data on such measures as research productivity, knowledge dissemination and 
transfer, training, international collaboration, and leveraging of financial resources. The 
information collected from these reports will allow SSHRC to better track the outputs of its 
research programs. The report will be tested as a pilot project in the late spring of 2002, 
and SSHRC plans to officially launch the report in late June 2002. Such final report forms 
are used by certain foreign agencies (e.g., the Economic and Social Research Council in 
the United Kingdom). 


The agencies suggest that measuring the outcomes or impact of some types of 
research is difficult since there are often long gaps between funding a research project or 
program and witnessing the socio-economic impact of that research. The Committee 
appreciates this issue, but it notes that in that case, the agencies should place more effort 
into long-term evaluations of the impact of their overall research programs through 
“retrospective analysis,” in addition to more short-term monitoring efforts. One witness 
argued that in addition to the time lag problem associated with measuring outcomes, 
there is also a problem associated with tracking the outputs and outcomes over time: 


For science and technology, tracking this output data over long periods of time is 
difficult. ... Research gets conducted in a given organization. It evolves into 
technology development. That may be conducted in another organization, [and] ... 
may be sponsored by another sponsor. ... It keeps going like that to eventual 
application ... The point is, it is very difficult to track research that was sponsored 
and performed by organizations initially and track them into eventual applications. 
[Ronald N. Kostoff, 88:11:10] 


% The report is accessible electronically from the Treasury Board of Canada Secretariat web site: http://www.tbs- 


sct.gc.ca/rma/dpr/00-01/NSERCOOdpre.pdf 
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| The Committee recognizes the difficulties associated with assessing the outcomes 
and impacts of agency funded research, especially in areas such as the social sciences 
and humanities. However, it believes that the granting agencies should do as much as 
possible to link budget with performance, and to better explain the economic, societal or 
environmental impact of research that they fund. Reporting on the qualitative and 
quantitative outputs, outcomes and impacts of agency-supported research should form 
part of Departmental Performance Reports, internal program evaluations, and material for 
public relations. The agencies have a responsibility to provide evidence to the 
government and taxpayer that there is “value for money” in the relatively large 
investments made by the government in the agencies. Such reports could also serve a 
role in helping the agencies to decide the research areas where funds should be 
allocated in the future. The Committee commends efforts made by the agencies to 
Improve performance monitoring, but believes that they can do more in this regard. To 
that end, the Committee recommends: 


RECOMMENDATION 11 


That the Government of Canada ensure that the federal granting 
agencies take steps to better measure and report on the outcomes 
and, where possible, impacts of their research programs for the 
benefit of the general public. 


Alternatives to Peer Review 


The Committee heard evidence that peer review is the most efficient system 
available for determining the allocation of federal research funds. Some witnesses 
indicated that without peer review, science of inferior quality might be supported by the 
granting agencies: 


An absence of peer review or ever watered down peer review may result in 
mediocre science, waste of resources, and in the long term, poor policy decisions. 
And some of my international colleagues have in fact reflected on this absence of 
peer review in certain areas of science and the problems its created. [Peter 
Johnson, Chair, Canadian Polar Commission, 75:09:15] 


However, the Committee also heard the opinion that the peer-review process 
needs to be improved, or even overhauled. A variety of alternatives to peer review have 
been proposed in recent years, including such mechanisms as productivity-based formula 
funding, bibliometrics, cash prizes, lottery, block grants to universities (where decision- 

h funding agencies to universities), discretionary (or 


ing is transferred from researc | | 
a funding, and bicameral review. Three of the leading alternatives, which were 
discussed during the Committee's hearings, are considered in more detail here. 


ET 


Bibliometrics 


Some researchers suggest that bibliometrics, either through the analysis of citation 
data or “journal impact factors” (a measure of the average number of citations earned by 
the papers that each journal contains), could be used as a supplement to the peer-review 
process: 


The other way bibliometrics can be useful is to aid peers in their decision to fund 
research. | would add that it does not replace peers’ judgment, but it can be used as 
a tool to help researchers, because bibliometrics can aid researchers in telling them 
what is the quality of the journals they evaluate in which researchers publish. [Benoit 
Godin, Observatoire des sciences et des technologies, 66:09:25] 


Using journal impact factors as an information tool to assess and compare the quality of 
an individual’s publishing record is a controversial issue since the impact factors were 
not designed to evaluate the work of individuals.” Additionally, extra cost and time is 
introduced when bibliometrics becomes a part of the evaluation process: 


Also, once we start using bibliometrics information as an explicit part of the process, 
then you actually have to start commitment funding to obtaining that information, 
ensuring its accurate and reliable, ensuring that the way it’s actually used by, for 
example, your review panel, it’s consistent across panels, documenting where there 
are problems, how you approach those problems and resolve them. So it’s an order 
of complexity that’s probably beyond a number of funding agencies at this time. 
[Fiona Wood, University of New England, Australia, 79:20:20] 


In 1999, the Medical Research Council (MRC) in the United Kingdom launched a 
pilot study to examine the feasibility of incorporating bibliometrics into its peer review 
process. The pilot study found that there was a good correlation between the results of 
bibliometric assessment and conventional peer review of past progress. However, the 
correlation of bibliometric assessments with the final award decision was quite low. The 
MRC concluded that bibliometrics should not be used routinely as part of the Council's 
evaluation procedures since the costs and extra review time needed to use the data 
properly would not be justified by the benefits offered.® The inclusion of bibliometric data 
as part of the peer-review process has been adopted by the Wellcome Trust, the world’s 
largest medical research charity. The Trust's neuroscience panel (annual budget of 
approximately £20 million or about $45 million) uses modified journal impact factors, 
citation analysis and paper counts, to help panel members judge the scientific record of 
applicants. These data are not used in isolation, and in fact, applicants judged to have 
excellent track records have been turned down for funding for other reasons (e.g., a poor 
research proposal).* The Committee encourages the Canadian granting agencies to 


** D. Adam, “The Counting House”, Nature, Vol. 415, February 2002, p. 726-29. 


$ See MRC Bibliometric Analyses Pilot Study, 1999, http:/Awww.mrc.ac.uk/index/funding/funding- 


specific_schemes/funding-evaluation_of_schemes/funding-bibliometric_analyses_pilot_study.htm 


°°  G. Lewison, R. Cottrell and D. Dixon, “Bibliometric indicators to assist the peer review process in grant 


decisions,” Research Evaluation, Vol. 8, April 1999, p. 47-52. 
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Bicameral Review 


One witness told the Committee that the present system for allocating research 
funds is completely flawed, and presented some anecdotal evidence to support his claim. 


The witness presented a proposal for an alternative peer review system, called “bicameral 
review”: 


It might be thought that current peer review procedures, despite their flaws, are 
better than simply allocating funds by tossing a coin. But coin tossing at least gives 
excellence a fighting change. In fact, the current system is worse than coin tossing 
since it actively selects against excellence ... Under bicameral review the first 
decision is made by the committee of peers, who only review the applicant's track 
record, not the applicant’s proposed project. The second decision is made in-house 
by specialists in the funding agency, who with respect to budget justification only 
review the applicant's proposed project, not the applicant's track record. [Donald 
Forsdyke, Professor, Queen’s University, 58:09:50] 


Under the system proposed by the witness, track record is assessed as a ratio of 
achievement to funds received by the peer review committee. The agency takes the 
applicant ratings provided by the peer-review committees and then decides what funds 
the applicant needs. Funds are allocated on a sliding scale: applicants at the top of the 
scale get 100% of what they are deemed to need, and applicants just below the top 
receive a lower proportion of funds. This allocation method progresses to the bottom of 
the scale, where the applicant may receive only 10% of what he or she needs. 


Productivity-based Formula Funding 


This alternative to peer review is based on the assumption that past success is the 
best predictor of future performance. Productivity-based formula funding proposes that 
researchers be funded based on their track records. Under such a system, funds would 
be allocated according to an algorithm (i.e., dollars awarded would be proportional to 
some weighted sum of numbers of publications, numbers of advanced degrees awarded, 


etc.). 


One witness pointed out that there are strong similarities between bicameral 
review and productivity-based formula funding: 


j hers 
rnatives place heavy emphasis on awards to established researc. 
oe is although they differ in how the track records would be 
the use of true technical experts in the evaluation of the 
onald Kostoff, 88:10:10] 
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The witness went on to suggest that since traditional peer review also places heavy 
emphasis on the track record of the performer(s) in reaching a funding 
recommendation, the two proposed alternatives are, in practice, fairly similar to 
standard peer review methods. The major difference is the absence of technical experts 
to evaluate the proposed research described in grant applications. 


The Committee welcomes concrete suggestions for changes to the system for the 
allocation of federal research funds. It has some concerns, however, about the bicameral 
review and productivity-based formula funding proposals. First, the lack of evaluation of 
the quality and feasibility of the proposed research itself by an expert panel is worrisome, 
especially in the context of project- (vs. program) based proposals. Second, the idea of 
giving money to every applicant under the bicameral review system is a concern given 
that funds are scarce, and funds would be channelled from applicants ranked highly by 
the initial review panel to lower-ranked applicants. The Committee does not believe that 
this method is an efficient or wise way to allocate taxpayers’ money. Nevertheless, the 
Committee encourages the agencies to review these and other proposed alternatives (or 
enhancements) to peer review when conducting internal evaluations of peer review (see 
recommendation 10). 


Although many alternatives to peer review have been proposed, most members of 
the research community do not consider that these alternatives could entirely replace 
peer review since many of them lack an arm’s length, “quality assurance” component. 
Rather, the general consensus is that peer review can be fine-tuned but not replaced: 


There really aren't alternatives to what funding agencies are using by way of 
obtaining scientific expertise for their decision-making. It’s really things like whether 
or not bibliometrics has a role to play in funding agencies in helping with the 
process. [Fiona Wood, University of New England, Australia, 79:20:20] 


My bottom line is that while peer review has its imperfections and limitations, there is 
little evidence that the best researchers and ideas are going without funding and far 
less evidence that the alternatives described above would improve the situation. 
[Ronald Kostoff, 88:10:10] 


The Committee concurs and believes that peer review is the most efficient method for 
determining the allocation of federal research funds. However the system can and 
should be improved. Regular reviews and refinements of peer review practices by the 
agencies themselves are critical to ensuring that the system for the allocation of federal 
research funds is efficient, transparent and responsive to the changing needs of the 
research community and other stakeholders. 
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After hearing from a number of witnesses representing various interests in the 
research community, the Committee believes that the mechanisms used to allocate 
research funds by the three federal granting agencies are, for the most part, efficient. 
Evidence the Committee heard, and data it collected on the nationwide distribution of 
granting agency funds, indicated that there are regional differences in the success rates 
of researchers in obtaining research funds and in the average size of grants received. 
Such differences were also noted in relation to the size of institutions, with researchers 
from larger institutions generally having a higher success rate in obtaining federal 
research funds and receiving larger grants, on average, than researchers from small 
institutions. The Committee believes that regional differences in success rates and 
funding levels likely reflect the relatively large number of small universities and low levels 
of value-added industrial activity and/or provincial investment in R&D in certain regions of 
the country. Researchers at small universities face barriers such as high teaching loads 
and small graduate programs that reduce their capacity to conduct research relative to 
their large university counterparts. 


Based on the evidence heard, the Committee is of the opinion that the best way to 
address some of the regional differences in success rates and grant levels in granting 
agency competitions is to first address differences in research capacity at institutions 
across the country, not to change the decision-making processes for the allocation of 
funds at the granting agencies. The Committee’s recommendations for the government to 
establish a permanent program for the support of the indirect costs of federally supported 
research, and to provide programs targeted to building research capacity at small or 
regional universities aim to “level the playing field” for researchers competing in open 
competitions at the granting agencies. Furthermore, the Committee encourages the 
granting agencies to review their funding mechanisms to ensure that researchers from 
these institutions participate actively in decision-making processes, and that, as far as 
possible, the peer-review process takes into account the difficulties faced by researchers 
at these institutions. The Committee also believes that low overall funding levels to the 
federal granting agencies have contributed to the generally low success rates and/or 
grant sizes for researchers from small and regional institutions. The Committee 
recommends that the government act quickly on the Innovation Strategy s commitment to 
increase funding levels to the granting agencies, especially to SSHRC. 
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properly supported. Furthermore, the Committee recommends that the agencies establish 
a formal mechanism to facilitate collaboration on all levels so that the research 
community can have seamless access to federal research funds in Canada. The 
Committee also recommends that the federal government examine the ways it sets S&T 
policy and decides on funding priorities. It believes that a stronger, more co-ordinated 
S&T governance framework, that could include a Chief Scientific Adviser and/or a science 
advisory body that reports to Parliament, would ensure that research in target areas of 
national importance, as well as other types of research, receives appropriate levels of 
funding. 


The Committee is of the opinion that other players in Canada’s innovation 
framework may not be receiving the attention that they deserve from the federal granting 
agencies. Although all three agencies have core programs to support the training of highly 
qualified personnel, low overall funding levels to the agencies translate into relatively low 
success rates and stipend levels for most training programs. The Committee 
recommends that the government increase funding levels to the granting agencies so that 
they can correct these deficiencies. Another group of researchers that the Committee 
feels is undervalued by the granting agencies is college researchers. This group is 
beginning to contribute more and more to the development and technology transfer end 
of the innovation continuum. In recognition of this contribution, the Committee 
recommends that the government establish separate funding programs, through the 
granting agencies and/or other government bodies, to support the work of college 
researchers. The Committee believes that the granting agencies should continue to 
encourage college researchers to participate in their programs, but it feels that this group 
of researchers also requires separate funding programs whose excellence-based 
selection criteria reflect the special situation faced by researchers in the college 
environment. 


The Committee supports the practice of peer review as a mechanism for 
determining the allocation of federal research funds, but it believes that a number of 
improvements can be made to the system. The Committee’s recommendations include 
making improvements to feedback or appeal mechanisms for applicants, and requiring 
the agencies to engage in more regular reviews and refinements of their peer-review 
processes and practices in general. Finally, the Committee believes that although it is 
important to examine the mechanisms employed by the agencies for the allocation of 
federal research funds, more effort should be placed into examining the outcomes and 
impacts of federally funded research. The Committee makes a recommendation to the 
government that it ensure that the granting agencies take steps to better measure these 
outcomes. This information would help the agencies decide on research areas that 
should receive special consideration for funding, and would present the government and 
public with a clear idea of the “value for money” of these granting programs. 
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APPENDIX 1 


| 


Gross Domestic Expenditures on R&D (GERD) as a Percentage of Gross Domestic Product for 
Selected OECD Countries’ 


Country 1991 1992 1993 1994 1995 1996 1997 1998 1999 
"Canada =s—“‘<‘izw OS 168. 1404400 CO ae 
France 2.41 242 2405 2.349002 30 2.39 222 2:18 217 
Germany 201 248° 2810220887 2002 2.29 2:31 2.44 
Italy 1.24 1120 1:18 1.05 1.00 1.01 0.99 1.02 1.04 
Japan’ 3:00 2.95 2.88) 2:84, S2:08mN ee 2.90 3.04 3.04 
Sweden 2.89 3:27 3.46 3.67 3.80 
UK 2.11 2118 - 2428002; 07 1.98 1.91 1.84 4.83 1.87 
U.S.A‘ 2.81 214 “2628. 242 02 56 2.54 2.07 2.60 2.64 
out RU i, Statistics, Vol. 25, No. 8, November 2001 (Source: Main Science 


} isti I Iletin Science 
Statistics Canada, Service Bu 
and Technology indicators, No. 4, DSTI, OECD, 2001). 
2 Overestimated or based on overestimated data. 
3  Underestimated or based on underestimed data. 


4 Excludes most or all capital expenditure. 


63 


"era: et F4 Se ie où mt 07 Vote À 
3B tac PS Tee oe MERE | 
DR v SR 


CC | 


Pi ew (Un tien Ts! 
Es PRE rer RES CRT EN , 
Ar 


Cog ARE Re ON. EUR a on 
Tee MAT NI NT NT NS 
rr ans So one ies . be TAS; = 24 Wal 


ENS 
4 


.  ~< | t | — és ES | aS 

Aad Ale ER 54 Spuus of, eal Vier be gi 

mé HET See yan ha tCrotmst 
ew , ar + de 1 Om Lac fee ee Tr 

be PP Cte ic Une 4 rh (RE dv | 

, ba . A FL Le “yitakisn, rat ‘ le ; (À 

RUE Rat a tuniion on he 

pavecrtor, Fes, i dupport Fin Le 

ey duc Ye gérilg nue ANGLE à con 1 a L 

HE CR uk propane, bis fects | her al 

Wasa (Un Dre | wiles CONTE 

hs lie faced En IPSSRTONES a a 

ls et 


= a 
ne 


es D. pod 


| | sors Ga hate bates. review uc ‘eme 
7 ref e À leis) déni tr Seeds, CT telever ara 
a a be sero. The Currie recommenda 
7 : ee Maui madiirmns ter applicants, and te 
A > Reh RE are! ens of thers 
a ae, “yee, Male dé Cocniree beaeles thal at 
'RPETE NX Due dail ws ÉD nse dle Gr Ce 
Lo. en om As "RoR pag en eng M 
Mau. a A à LT à: + Re 
ere bet nb ee 
CONTES ET, i 
eet rh tae < 
, utah. A à eat Aa bre ia es 
LE ne à PE 


: 0 © LR F 
L : 


+ 


APPENDIX 2 


APPENDIX 2A — Success Rates and Average Annual Grant Size for NSERC Applicants 
(Competition Year 2001') 


Success Rate by Size of Institution 


Success Rate by Region 


@ 

(=) 

@ 

oO 
i 


80 - 80 - 
40 - pees, 
60 | 2600 
2 2 
Po 50 - o 50 
% 40. ÿ 40 - 
o © 
8 30 - iy YO) = 
> =] 
7200.4 F0 
Ok 10: 
0 0 
LARGE MEDIUM SMALL  OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Average Grant Size by Size of Institution Average Grant Size by Region 
45000 - 45000 - 
40000 - 40000 - 
& 35000 - & 35000 - 
o [0] 
4 .N 000 - 
À 30000 a 2000 
£ 25000 - £ 25000 
& 20000 - & 20000 - 
i) 
® 15000 - eg a 
® 10000 - 2 40000 
Z < 
5000 - 5000 - 
0 RAS ESS (6) = EN So be 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Amount Awarded by Size of Institution Amount Awarded by Region 
ATLANTIC 
SMALL $6.4M 
$12.1M (7%) WEST 
(13%) $26.5M 
(29%) 
QUEBEC 
LARGE $20.7M 
$43.7M (23%) 
(48%) 
MEDIUM 
$35.7M ONTARIO 
(39%) $37.9M 
(41%) 


EE 


‘ Raw data supplied by NSERC for its Research Grants Program. 
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APPENDIX 2B — Success Rates and Average Annual Grant Size 


for SSHRC Applicants (Competition Year 2001°) 


Success Rate by Size of Institution 


Success Rate by Region 


MEDIUM 
$6.3M 
(33%) 


(54%) 


50 50m 
45 - 45 
40 - ae 
se 35 | & 35 - 
© Scr = 30. 
2 œ 
ac 95 - 7 25 - 
7) 
8 20- $ 20 
9 15 - S15 - 
7) [2] 
107 102 
5 5 5-4 
(0) 0 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Average Grant Size by Size of Institution Average Grant Size by Region 
30000 - 30000 - 
=~ — 250008 
& 25000 & 5000 
N N 20000 
a 20000 - À : 
= = 
S_15000 S 15000 - 
G S 
D 10000 D 
8 ; D 10000 - 
5 g 
< 5000 - < 5000 - 
0 0 : : 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Amount Awarded by Size of Institution Amount Awarded by Region 
SMALL ATLANTIC 
$2.4M $1.1M 
( 1 3%) (6%) WEST 
$4.6M 
(24%) 
QUEBEC 
LARGE $5.7M 
$10.4M (30%) 


ONTARIO 
$7.8M 
(40%) 


Raw data supplied by SSHRC for its Standard Research Grants Program. 
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APPENDIX 2C — Success Rates and Average Annual Grant Size for CIHR Applicants 


(Competition Year 2001°) 
Success Rate by Size of Institution Success Rate by Region 
40 - 
35 - 
£ Q 30 4 
| £ @ 25- 
© œ 
” 7 20 a] 
rk U) 
© sus 
S © 
a ÿ 10 - 
5 4 
- : 0 2 EE 
LARGE MEDIUM SMALL  OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Average Grant Size by Size of Institution Average Grant Size by Region 
120000 - 120000 5 
— 100000 - — 100000 - 
# Le 
fe : 
5 80000 7 80000 + 
= € 
g 60000 - £ 60000 - 
(©) 6 
© & 
ë 40000 - e 40000 + 
En ® 
2 20000 | < 20000 - 
0 aia : is ams k ÿ 0 Ë 
LARGE MEDIUM SMALL OVERALL WEST ONTARIO QUEBEC ATLANTIC OVERALL 
Amount Awarded by Size of Institution | Amount Awarded by Region 
SMALL ATLANTIC 
$0.78M | $1.9M 
MEDIUM ° (2%) 
$15.8M 7 : WEST 
(18%) | QUEBEC $24.0M 
$29.5M (27%) 
(34%) 
| 
LARGE 
$71.1M ONTARIO 
(81%) $32.2M 
| | (37%) 


; 
3 Raw data supplied by CIHR for its Operating Grants Program. 
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Appendix 2D -— List of Universities by Region and by Size’ of Institution 


Region 


Westy TE  |Ontatio oes 2 oe toes] QUE mere eee 


UBC 
Alberta 
Calgary 
Manitoba 


Saskatchewan 
Simon Fraser 
Victoria 


Regina 
Lethbridge 
Winnipeg 
Brandon 
Northern BC 
Trinity Western 


Augustana Univ.Coll. 


Canadian Bible Coll. 
Athabasca 

Univ. Coll. Cariboo 
Coll. U. St. Boniface 


McMaster 
Ottawa 
Waterloo 
Queen's 
Guelph 
Carleton 
Ryerson 


Windsor 
Brock 

Wilfrid Laurier 
Laurentian 
Lakehead 
Trent 

RMC 
Nipissing 
King's College 
Huron College 
Redeemer College 


Laval 
McGill 
Montreal 


UQAM 
Sherbrooke 


Concordia New Brunswick 


Mount St. Vincent 
Mount Allison 

Univ. Coll. Cape Breton 
St. Thomas 

NS Ag. College 

Univ. of King's Coll. (NS) 


Univ. Coll. Fraser Valley |St. Jerome's 
Luther College Coll. Dominican 
Malaspina Univ. Coll. Brescia College 
Okanagan Univ. Coll. Sudbury 
Regent College St. Michael's Coll. 
Royal Roads Victoria (U. of T.) 
Tech. U. of B.C. Wycliffe Coll. 
Saint Paul 


Size of institution: Small = 0-499 faculty members; medium = 500-999 faculty members; and large = greater 
than 1000 faculty members; data on faculty numbers are from 1999 (most recent figures available) and were 
provided by the Association of Universities and Colleges of Canada. 
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APPENDIX 3A — Recent Evaluation Studies and Reviews at NSERC’ 


Program Evaluation Studies Year of Report 


Discovery Grants Program (formerly Research Grants) — Phase Il (Evaluation) On-going 2001/2002 
Discovery Grants Program (formerly Research Grants) — Phase | (Design) 2000 
Strategic Projects Program (formerly Strategic Projects Grants) 1999 
Technology Partnerships Program 1998 
Women's Faculty Awards Program 1997 
University-Government Program . 1994 
Scholarships and Fellowships Programs (PGS, PDF, USRA) 1993 
Follow-up to Research Partnerships 1992 
Special Microelectronics Fund 1991 
Research Partnerships Programs (IRC, CRP, CRD) 1991 
Research Tools and Instrument Grants (formerly Equipment Grants) 1990 
Strategic Projects Grants 1988 
Canadian Microelectronics Corporation 1988 
Discovery Grants Program (formerly Operating Grants) and Steacie Fellowships 1986 
Program 

Scholarships and Fellowships Programs (PGS, PDF, USRA) 1985 


Pre-evaluation/Framework Study 


Discovery Grants Program (formerly Research Grants) 1999 
Technology Partnerships Program (framework) 1995 
Scholarships and Fellowships Programs (PGS, PDF, USRA) 1991 
Research Partnerships Programs (IRC, CRP, CRD) 1990 
Research Tools and Instrument Grants 1989 
Strategic Projects Grants 1987 
Discovery Grants Program (formerly Operating Grants) 1985 


Joint Evaluation Studies (Tri-Council/Departmental) 


On-going 2001/2002 
On-going 2001/2002 
On-going 2002 


Chairs in the Management of Technological Change 
Networks of Centres of Excellence Program 
Canada Research Chairs 

Networks of Centres of Excellence Program — Phase Il eel 


Eco-research Program a 
Canadian Genome Analysis and Technology Program 


im Evaluati 1993 
Networks of Centres of Excellence Program (Interim Evaluation) 
Networks of Centres of Excellence Program - Phase | (Evaluation Assessment) = 
Matching Funds 
Other Studies 
ili hips Programs On-going 2002 
Results-based Accountability Framework for Research Partnership g eh 9 


Grants Adjudication Analysis (peer review) 


Loo A eee 
: Data supplied by NSERC. 
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APPENDIX 3B — Recent Evaluation Studies and Review Activities at SSHRC? 


Program Evaluation Studies 


Immigration and the Metropolis Joint Initiative 

Women and Change Strategic Theme Program 

Review of SIGs and Aid to Small Universities programs 
Health Promotion Joint Initiative 

Strategic Grants Program 

General Research Grants Program 

Postdoctoral Fellowships Program 

Dissemination Programs Cluster 

Support to Specialized Research Collections and Fleeting Opportunities 
Strategic Theme: Education and Work in a Changing Society 
International Summer Institutes Pilot Program 

Joint Initiative: Aboriginal Affairs 

Strategic Theme: Women and Work 

Doctoral Fellowships Program 

Major Grants Program 

Aid to Scholarly Publications Program 

Canadian Studies Research Tools Program 

Aid to Small Universities Program 

Research in Priority Areas (Thematic Research) 


Joint Evaluation Studies (Tri-Council/Departmental) 


Networks of Centres of Excellence 

Canada Research Chairs 

Chairs in the Management of Technological Change 

Networks of Centres of Excellence Program — Phase 2 
Networks of Centres of Excellence Program — Interim Evaluation 


Other Studies 


Review of SSHRC's Research Time Stipend Policy 

Peer review — Research Grants Program (Gaskell Report) 

Peer review — Research Grants Program 

Peer review — Strategic Grants Program 

Peer review — Doctoral Fellowships Program 

Peer review — Research Grants Program 

Peer review — Research Grants Program 

Needs & Priorities Study for Reorientation of the Evaluation Function 


Data supplied by SSHRC. 
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Year of Report 


2001 
2000 
2000 
1998 
1996 
1995 
1994 
1994 
1994 
1993 
1993 
1992 
1992 
1991 
1991 
1991 
1991 
1989 
1989 


On-going 2001/2002 
On-going 2002 
2002 
1997 
1993 


2002 
1999 
1995 
1994 
1993 
1992 
1991 
1991 


APPENDIX 3C — Recent Evaluation Studies and Activities at MRC and CIHR* 


Program Evaluation Studies 


CIHR Operating Grants Program 

MRC-PMAC Health Program 

International Review of the MRC 

NHRDP-MRC Joint Program for the Development of Research in Nursing 


Pre-evaluation/Framework Study 


Rx&D-CIHR Research Program 

CIHR Operating Grants Program 

CIHR Performance Measurement Framework 
MRC Performance Indicators Framework 


Joint Evaluation Studies (Tri-Council/Departmental) 


Networks of Centres of Excellence Program 

Canada Research Chairs Program 

Networks of Centres of Excellence Program - Phase 2 
Canadian Genome Analysis and Technology Program 
Canadian Breast Cancer Research Initiative 

Eco-research Program 

Networks of Centres of Excellence Program (Interim Evaluation) 


Other Studies 


Stakeholder Satisfaction Surveys - Service Improvement Initiative 

CIHR Performance Measurement Strategy 

CIHR Planning Reporting and Accountability Structure 

System for selecting MRC Doctoral Research Awards recipients 

Review of a Study of Journal Impact Scores for Selected Groups of MRC Grant 
Applicants . 

Performance Indicators for the Technology Partnerships Program . | 
Maintaining the Momentum, Health Research as a Growth Sector in the Canadian 


Economy. Input to Program Review 


Re 
| Data supplied by CIHR. 
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Year of Report 


On-going 2002 
1999 
1996 
1993 


2002 
2002 
2001 
1997 


On-going 2001/2002 
On-going 2002 
1997 
1996 
1996 
1996 
1993 


2002 
2002 
2001 
1996 
1997 


1995 
1994 
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APPENDIX 4 — WITNESSES 


Organizations Appeared Meeting No. 
Advisory Council on Science and Technology 22/11/2001 55 


Jacquelyn Thayer Scott, President and Vice-Chancellor, 
University College of Cape Breton 


Alberta Heritage Foundation for Medical Research 07/02/2002 66 
Matthew Spence, President and Chief Executive Officer 


Association of Canadian Community Colleges 25/10/2001 43 
Gerald Brown, President and Chief Executive Officer 


Association of Universities and Colleges of Canada 08/11/2001 ot 
Robert J. Giroux, President and Chief Executive Officer 
David Barnard, President, University of Regina 
Paul Davenport, President, University of Western Ontario 
Bonnie Patterson, President, Trent University 
Tom Traves, President, Dalhousie University 


Canada Research Chairs Program 16/10/2001 39 
René Durocher, Executive Director 
Canadian Association of University Research 25/10/2001 43 


Administrators 
Wayne Marsh, President 
Canadian Association of University Teachers 25/10/2001 43 


Michael Piva, Member of the Executive Committee 


Canadian Institutes of Health Research 16/10/2001 39 
Mark Bisby, Director, Research Portfolio 
Canadian Polar Commission 11/04/2002 75 
Peter Johnson, Chair 
Steven Bigras, Executive Director 
Centre for Policy Research on Science and Technology 07/02/2002 66 
(CPROST), Simon Fraser University 
Adam Holbrook, Associate Director and Adjunct Professor 
22/11/2001 55 


Council of Science and Technology Advisors 
Alan Winter, President and Chief Executive Officer 
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Organizations 


Individuals 


lan Dohoo, Professor, Epidemology, Associate Dean, 
University of P.E.I. 


Donald Forsdyke, Professor, Department of 
Biochemistry, Queen's University 


Peter Frise, Professor, Mechanical/Design Engineering, 
University of Windsor 


Ronald N. Kostoff, Expert in Research Evaluation 


Bryan Poulin, Professor, Faculty of Business 
Administration, Lakehead University 


Fiona Wood, Senior Research Fellow and Lecturer, 
Centre for Higher Education and Management Policy, 
School of Professional Development and Leadership, 
University of New England, Australia 


Natural Sciences and Engineering Research Council of 
Canada 


Elizabeth Boston, Director, Research Grants 
“Observatoire des sciences et des technologies” 
Benoit Godin, Professor, INRS 


Social Sciences and Humanities Research Council of 
Canada 


Ned Ellis, Vice-President, Programs 
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Appeared 


29/11/2001 


29/11/2001 


29/11/2001 


04/06/2002 


29/11/2001 


23/04/2002 


16/10/2001 


07/02/2002 


16/10/2001 


Meeting No. 


58 


58 


58 


88 


58 


TAs) 


39 


66 


39 


REQUEST FOR GOVERNMENT RESPONSE 


Pursuant to Standing Order 109, the Committee requests that the government table 
a comprehensive response to this report within one hundred and fifty (150) days. 


A copy of the relevant Minutes of Proceedings of the Standing Committee on 
Industry, Science and Technology (Meetings Nos. 39, 43, 51, 55, 58, 66, 67, 68, 75, 84, 
85, 86, 88, 89 and 90 which includes this report) is tabled. 


Respectfully submitted, 


Walt Lastewka, M.P. 
St. Catharines 


Chair 
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Canadian Alliance Dissenting Report 


The Canadian Alliance does Support prudent investments in innovation, technology 
and research. It has called for increasing funding for Canada’s research granting councils: 
the National Research Council (NRC), Natural Sciences and Engineering Research 
Council (NSERC), Social Sciences and Humanities Research Council (SSHRC), and the 
Canadian Institutes of Health Research (CIHR). The Canadian Alliance also supports a 


competitive peer review process to determine who shall receive grants through these 
respective councils. 


The Canadian Alliance members of this committee would like to emphasize certain 
recommendations of this report, as well as express some concerns or reservations about 
other recommendations. 


Firstly, the Canadian Alliance strongly supports recommendation 6: “That the 
Government of Canada establish a more formalized mechanism, in consultation with the 
provinces and territories, for setting or modifying S&T policy, deciding on funding priorities, 
and ensuring that they are implemented. Such a framework could include a science 
advisory body and/or Chief Scientific Adviser that would report directly to Parliament. 


We have consistently called for a funding framework for Science & Technology in 
the 37° Parliament; unfortunately, numerous Secretaries of State for Science and 
Technology and Ministers of Industry have ignored this advice and failed to establish a 
framework. We hope that the government will finally act on this recommendation. 


We also hope that the government will appoint a Chief Scientist of Canada, who 
would coordinate science activities in all government departments, help scientists 
communicate their findings, and help bridge the gap between scientists, bureaucrats and 


elected officials. 


Secondly, the Canadian Alliance is very concerned that over half of the 
recommendations in this report call for increased funding in one way or another, and do so 
in a very general way with no specific figures or guidelines attached. From our perspective, 
to constantly request more funding for every area, group, or agency without attaching 
specific numbers — or even prioritizing — IS fiscally irresponsible for a Parliamentary 
Committee and not the way in which to build a modern Research & Development 


infrastructure in Canada. 


iance believes that we should not single out SSHRC for 
ae Sa ements number 3. There was much debate about 
unding for SSHRC vis-a-vis the other granting agencies; 
be based on some evidence other than statements by the 
This could certainly be an issue for further study by this 


Thirdly, 
special attention, as is done 
the perceived inequity in f 
however, this perception must 
granting agencies themselves. 
committee. 
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Furthermore, tying increased funding to SSHRC to small universities is a mistake. If 
small universities are facing extraordinary funding problems, then the government should 
address this issue separately from the funding of research and development. It should 
address it through stable long-term funding via the Canada Health & Social Transfer 
(CHST), rather than setting up separate national programs — such as the recent Trudeau 
fellowship — which deplete the government's ability to ensure stable long-term funding. 


Fourthly, the Canadian Alliance is concemed that the committee is recommending 
moving away from merit as the guiding principle to direct research funding. As mentioned 
with respect to SSHRC, Committee members appear intent on using funding to granting 
agencies to address fiscal problems facing small universities and colleges. In order to be 
internationally competitive in science, medical and humanities research, Canada must be 
guided by merit and excellence in its public funding of Research & Development; and 
should address the problem of funding for small universities and colleges separately. 


The Canadian Alliance members of this Committee are concerned about the 
implications of funnelling research dollars into every post-secondary institution in Canada. 
Some committee members seem intent on transforming ever post-secondary institution in 
Canada into a research-based institution; and in doing so, seem unaware of the dangers 
this would pose to a true liberal arts education. One of the major crises facing 
post-secondary education in the humanities is pressure on professors to focus on 
research at the expense of teaching and developing the next generation of critical thinking 
citizens, citizens which are essential to ensuring the long term health of a democratic 
society. 


James Rajotte, Industry Critic 
Brian Fitzpatrick 
Cheryl Gallant 
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MINUTES OF PROCEEDINGS 


Thursday, June 6, 2002 
(Meeting No. 89) 


The Standing Committee on Industry, Science and Technology met in camera at 


9:02 a.m. this day, in Room 705, La Promenade Building, the Chair, Walt Lastewka, 
presiding. 


Members of the Committee present Stéphane Bergeron, Brian Fitzpatrick, Walt 
Lastewka, Serge Marcil, Joe McGuire, Dan McTeague, James Rajotte, Andy Savoy, 
Brent St. Denis and Joseph Volpe. 


In attendance: From the Library of Parliament. Lalita Acharya and James McQueen, 
Research Officers. 


Pursuant to Standing Order 108(2), the Committee resumed consideration of its study 
on the Three Federal Granting Agencies, Peer Review Funding, and the Canada 
Research Chairs Program (See Minutes of Proceedings, Tuesday, October 16, 2001, 
Meeting No. 39). 


It was agreed,— That the Committee authorize the printing of dissenting or 
supplementary opinions as an appendix to this report, immediately following the 
signature of the Chair. 


It was agreed, — That the opinions be limited to not more than five (5) pages. 


It was agreed, — That the opinions be received by the Clerk in both official languages 
no later than Monday, June 10, 2002 at 3:00 p.m. 


At 10:38 a.m., the Committee adjourned to the call of the Chair. 


Tuesday, June 11, 2002 
(Meeting No. 90) 


Industry, Science and Technology met at 9:10 a.m. this 


i ittee on di 
None enr k, the Chair, Walt Lastewka, presiding. 


day, in Room 269, West Bloc 
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Members of the Committee present. Larry Bagnell, Stéphane Bergeron, Brian 
Fitzpatrick, Chery! Gallant, Walt Lastewka, Serge Marcil, Joe McGuire, Dan McTeague, 
Andy Savoy and Brent St. Denis. 


Acting Members present. Pierre Brien for Stephane Bergeron. 


In attendance: From the Library of Parliament. Dan Shaw, Lalita Acharya and James 
McQueen, Research Officers. 


Witnesses: From the Department of Industry. Renée St-Jacques, Chief Economist and 
Director General, Micro-Economic Policy Analysis Branch; Someshwar Rao, Director, 
Micro-Economic Analysis Directorate. From Statistics Canada: John R. Baldwin, 
Director, Micro Economic Studies and Analysis; Tarek Harchaoui, Chief, Multifaction 
Productivity Measures, Micro-Economic Studies and Analysis. 


Pursuant to Standing Order 108(2), the Committee commenced its consideration of the 
subject of Innovation, Productivity and Growth. 


Renée St-Jacques and John Baldwin each made an opening statement and, with the 
other witnesses, answered questions. 


At 10:40 a.m., the Committee proceeded to sit in camera. 


Pursuant to Standing Order 108(2), the Committee resumed its consideration of its 
study on the Three Federal Granting Agencies, Peer Review Funding, and the Canada 
Research Chairs Program (See Minutes of Proceedings, Tuesday, October 16, 2001, 
Meeting No. 39). 


It was agreed, — That the draft report (as amended) be concurred in. 


Ordered, — That the Chair present the Tenth Report (as amended) to the House at the 
earliest possible opportunity. 


It was agreed, — That pursuant to Standing Order 109, the Committee request that the 
Government table a comprehensive response to this report within one hundred fifty 
(150) days. 


It was agreed, — That the Chair be authorized to make such typographical and editorial 
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pase as may be necessary without changing the substance of the draft report to the 
ouse. 


It was agreed, — That 550 copies of the Report be printed in a tumble format. 


At 11:00 a.m., the Committee adjourned to the call of the Chair. 


Normand Radford 
Clerk of the Committee 
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